Change in eigenvalues due to perturbation to a correlation matrix
$begingroup$
Let $A$ be a $m times n$ matrix defined as
$ A = Big[frac{a_1}{|a_1|} cdots frac{a_n}{|a_n|}Big]$ and $a_k in mathbb{R}^{mtimes 1}$ where $k in [1,dots,n]$.
Now, we define a correlation matrix $R = A^TA$ where each diagonal element is $1$ and it is a symmetric matrix. The trace of $R$, i.e., $mathbb{Tr}(R) = n$.
All non-diagonal elements of $R$ represents the correlation among the columns of $A$. We define them by correlation-coefficients
$rho_{jk} = Big(frac{a_j}{|a_j|}Big)^TBig(frac{a_k}{|a_k|}Big)$
which satisfy $-1 leq rho_{jk} leq 1$.
In my present work, I modify each column of $A$ such that the correlation among the columns of $A$ increases. Consequently, the correlation-coefficients also increases proportionally in $R$. I am interested to comment on the change in eigenvalues of $R$ with increase in correlation-coefficients.
Numerically, I observed that only largest eigenvalue of $R$ increases whereas rest of the eigenvalues decreases. But, I am unable to verify this phenomenon theoretically. Therefore, I ask you here for a hint to proceed my investigation further.
More precisely, the claim is:
Let $lambda$ be the set of eigenvalues of $R$ where $lambda_1 geq cdots geq lambda_n geq 0$ and $hat{lambda}$ be the set of eigenvalues of $hat{R}$ where $hat{lambda}_1 geq cdots geq hat{lambda}_n geq 0$. Assume that the correlation-coefficients in $hat{R}$ satisfy
$$
hat{rho}_{jk} geq rho_{jk} quad {j,k} in [1,cdots,n] quad text{and} quad jneq k.
$$
Moreover the trace of correlation matrices remains same, i.e.,
$$
mathbb{Tr}(hat{R}) = mathbb{Tr}({R}) = n.
$$
Consequently, we claim that the eigenvalues of $hat{R}$ and $R$ satisfy the following inequalities:
$$
hat{lambda}_1 geq lambda_1 quad text{and} quad hat{lambda}_i leq lambda_i quad iin[2,dots,n].
$$
Example:
Suppose, all columns of $A$ are orthonormal. This implies that the resulting correlation matrix would be an identity matrix and in this case, all eigenvalues are equal to $1$.
Now, suppose all columns are linearly dependent by a positive factor. This implies that the all correlation-coefficients is equal to 1 and the resulting correlation matrix is a rank-1 matrix, i.e., $mathbb{1}mathbb{1}^T$ where the largest eigenvalue is $n$ and rest of the eigenvalues are zero.
In this example, the largest eigenvalue increases from $1$ to $n$ when correlation matrix changes from the identity matrix to the rank-1 matrix. On the other hand, rest of the eigenvalues decreases from $1$ to $0$.
In order to prove the above mentioned claim, will it be sufficient:
if we can show that the largest eigenvalue path from the identity matrix to a matrix of ones, i.e., $mathbb{1}mathbb{1}^T$ is monotonically non-decreasing. Here, we will change only off-diagonal elements which always lies between -1 to 1. We also establish similar behaviour for rest of the eigenvalues ?
Can you comment on this approach? If you think, it could be a right direction. Do you have any suggestion how should I start to prove/disprove the claim?
linear-algebra matrices eigenvalues-eigenvectors linear-transformations perturbation-theory
asked Dec 9 '18 at 8:38
harihari
63
$endgroup$
add a comment |
$begingroup$
Let $A$ be a $m times n$ matrix defined as
$ A = Big[frac{a_1}{|a_1|} cdots frac{a_n}{|a_n|}Big]$ and $a_k in mathbb{R}^{mtimes 1}$ where $k in [1,dots,n]$.
Now, we define a correlation matrix $R = A^TA$ where each diagonal element is $1$ and it is a symmetric matrix. The trace of $R$, i.e., $mathbb{Tr}(R) = n$.
All non-diagonal elements of $R$ represents the correlation among the columns of $A$. We define them by correlation-coefficients
$rho_{jk} = Big(frac{a_j}{|a_j|}Big)^TBig(frac{a_k}{|a_k|}Big)$
which satisfy $-1 leq rho_{jk} leq 1$.
In my present work, I modify each column of $A$ such that the correlation among the columns of $A$ increases. Consequently, the correlation-coefficients also increases proportionally in $R$. I am interested to comment on the change in eigenvalues of $R$ with increase in correlation-coefficients.
Numerically, I observed that only largest eigenvalue of $R$ increases whereas rest of the eigenvalues decreases. But, I am unable to verify this phenomenon theoretically. Therefore, I ask you here for a hint to proceed my investigation further.
More precisely, the claim is:
Let $lambda$ be the set of eigenvalues of $R$ where $lambda_1 geq cdots geq lambda_n geq 0$ and $hat{lambda}$ be the set of eigenvalues of $hat{R}$ where $hat{lambda}_1 geq cdots geq hat{lambda}_n geq 0$. Assume that the correlation-coefficients in $hat{R}$ satisfy
$$
hat{rho}_{jk} geq rho_{jk} quad {j,k} in [1,cdots,n] quad text{and} quad jneq k.
$$
Moreover the trace of correlation matrices remains same, i.e.,
$$
mathbb{Tr}(hat{R}) = mathbb{Tr}({R}) = n.
$$
Consequently, we claim that the eigenvalues of $hat{R}$ and $R$ satisfy the following inequalities:
$$
hat{lambda}_1 geq lambda_1 quad text{and} quad hat{lambda}_i leq lambda_i quad iin[2,dots,n].
$$
Example:
Suppose, all columns of $A$ are orthonormal. This implies that the resulting correlation matrix would be an identity matrix and in this case, all eigenvalues are equal to $1$.
Now, suppose all columns are linearly dependent by a positive factor. This implies that the all correlation-coefficients is equal to 1 and the resulting correlation matrix is a rank-1 matrix, i.e., $mathbb{1}mathbb{1}^T$ where the largest eigenvalue is $n$ and rest of the eigenvalues are zero.
In this example, the largest eigenvalue increases from $1$ to $n$ when correlation matrix changes from the identity matrix to the rank-1 matrix. On the other hand, rest of the eigenvalues decreases from $1$ to $0$.
In order to prove the above mentioned claim, will it be sufficient:
if we can show that the largest eigenvalue path from the identity matrix to a matrix of ones, i.e., $mathbb{1}mathbb{1}^T$ is monotonically non-decreasing. Here, we will change only off-diagonal elements which always lies between -1 to 1. We also establish similar behaviour for rest of the eigenvalues ?
Can you comment on this approach? If you think, it could be a right direction. Do you have any suggestion how should I start to prove/disprove the claim?
linear-algebra matrices eigenvalues-eigenvectors linear-transformations perturbation-theory
asked Dec 9 '18 at 8:38
harihari
63
$endgroup$
$begingroup$
It is not clear to me if your allowed perturbation preserves the $a_i/|a_i|$ condition. If it does, that you know that the trace is still $n$ under the perturbation and hence the sum of eigenvalues is unchanged. In the 2d example that you presented, How do you know that $delta>0$? A negative $delta$ would invalidate your claim.
$endgroup$
– user617446
Dec 9 '18 at 12:53
$begingroup$
@user617446 Yes, the allowed perturbation preserves the $a_i/|a_i|$ condition. This is always ensured. You are right, in this condition, sum of eigenvalues is unchanged and somehow I should utilize this information to prove my arguments. In the 2nd example, $delta$ is positive by construction. This signify that the correlation among all columns of $A$ are increasing.
$endgroup$
– hari
Dec 9 '18 at 15:32
add a comment |
$begingroup$
Let $A$ be a $m times n$ matrix defined as
$ A = Big[frac{a_1}{|a_1|} cdots frac{a_n}{|a_n|}Big]$ and $a_k in mathbb{R}^{mtimes 1}$ where $k in [1,dots,n]$.
Now, we define a correlation matrix $R = A^TA$ where each diagonal element is $1$ and it is a symmetric matrix. The trace of $R$, i.e., $mathbb{Tr}(R) = n$.
All non-diagonal elements of $R$ represents the correlation among the columns of $A$. We define them by correlation-coefficients
$rho_{jk} = Big(frac{a_j}{|a_j|}Big)^TBig(frac{a_k}{|a_k|}Big)$
which satisfy $-1 leq rho_{jk} leq 1$.
In my present work, I modify each column of $A$ such that the correlation among the columns of $A$ increases. Consequently, the correlation-coefficients also increases proportionally in $R$. I am interested to comment on the change in eigenvalues of $R$ with increase in correlation-coefficients.
Numerically, I observed that only largest eigenvalue of $R$ increases whereas rest of the eigenvalues decreases. But, I am unable to verify this phenomenon theoretically. Therefore, I ask you here for a hint to proceed my investigation further.
More precisely, the claim is:
Let $lambda$ be the set of eigenvalues of $R$ where $lambda_1 geq cdots geq lambda_n geq 0$ and $hat{lambda}$ be the set of eigenvalues of $hat{R}$ where $hat{lambda}_1 geq cdots geq hat{lambda}_n geq 0$. Assume that the correlation-coefficients in $hat{R}$ satisfy
$$
hat{rho}_{jk} geq rho_{jk} quad {j,k} in [1,cdots,n] quad text{and} quad jneq k.
$$
Moreover the trace of correlation matrices remains same, i.e.,
$$
mathbb{Tr}(hat{R}) = mathbb{Tr}({R}) = n.
$$
Consequently, we claim that the eigenvalues of $hat{R}$ and $R$ satisfy the following inequalities:
$$
hat{lambda}_1 geq lambda_1 quad text{and} quad hat{lambda}_i leq lambda_i quad iin[2,dots,n].
$$
Example:
Suppose, all columns of $A$ are orthonormal. This implies that the resulting correlation matrix would be an identity matrix and in this case, all eigenvalues are equal to $1$.
Now, suppose all columns are linearly dependent by a positive factor. This implies that the all correlation-coefficients is equal to 1 and the resulting correlation matrix is a rank-1 matrix, i.e., $mathbb{1}mathbb{1}^T$ where the largest eigenvalue is $n$ and rest of the eigenvalues are zero.
In this example, the largest eigenvalue increases from $1$ to $n$ when correlation matrix changes from the identity matrix to the rank-1 matrix. On the other hand, rest of the eigenvalues decreases from $1$ to $0$.
In order to prove the above mentioned claim, will it be sufficient:
if we can show that the largest eigenvalue path from the identity matrix to a matrix of ones, i.e., $mathbb{1}mathbb{1}^T$ is monotonically non-decreasing. Here, we will change only off-diagonal elements which always lies between -1 to 1. We also establish similar behaviour for rest of the eigenvalues ?
Can you comment on this approach? If you think, it could be a right direction. Do you have any suggestion how should I start to prove/disprove the claim?
linear-algebra matrices eigenvalues-eigenvectors linear-transformations perturbation-theory
asked Dec 9 '18 at 8:38
harihari
63
$endgroup$
Let $A$ be a $m times n$ matrix defined as
$ A = Big[frac{a_1}{|a_1|} cdots frac{a_n}{|a_n|}Big]$ and $a_k in mathbb{R}^{mtimes 1}$ where $k in [1,dots,n]$.
Now, we define a correlation matrix $R = A^TA$ where each diagonal element is $1$ and it is a symmetric matrix. The trace of $R$, i.e., $mathbb{Tr}(R) = n$.
All non-diagonal elements of $R$ represents the correlation among the columns of $A$. We define them by correlation-coefficients
$rho_{jk} = Big(frac{a_j}{|a_j|}Big)^TBig(frac{a_k}{|a_k|}Big)$
which satisfy $-1 leq rho_{jk} leq 1$.
In my present work, I modify each column of $A$ such that the correlation among the columns of $A$ increases. Consequently, the correlation-coefficients also increases proportionally in $R$. I am interested to comment on the change in eigenvalues of $R$ with increase in correlation-coefficients.
Numerically, I observed that only largest eigenvalue of $R$ increases whereas rest of the eigenvalues decreases. But, I am unable to verify this phenomenon theoretically. Therefore, I ask you here for a hint to proceed my investigation further.
More precisely, the claim is:
Let $lambda$ be the set of eigenvalues of $R$ where $lambda_1 geq cdots geq lambda_n geq 0$ and $hat{lambda}$ be the set of eigenvalues of $hat{R}$ where $hat{lambda}_1 geq cdots geq hat{lambda}_n geq 0$. Assume that the correlation-coefficients in $hat{R}$ satisfy
$$
hat{rho}_{jk} geq rho_{jk} quad {j,k} in [1,cdots,n] quad text{and} quad jneq k.
$$
Moreover the trace of correlation matrices remains same, i.e.,
$$
mathbb{Tr}(hat{R}) = mathbb{Tr}({R}) = n.
$$
Consequently, we claim that the eigenvalues of $hat{R}$ and $R$ satisfy the following inequalities:
$$
hat{lambda}_1 geq lambda_1 quad text{and} quad hat{lambda}_i leq lambda_i quad iin[2,dots,n].
$$
Example:
Suppose, all columns of $A$ are orthonormal. This implies that the resulting correlation matrix would be an identity matrix and in this case, all eigenvalues are equal to $1$.
Now, suppose all columns are linearly dependent by a positive factor. This implies that the all correlation-coefficients is equal to 1 and the resulting correlation matrix is a rank-1 matrix, i.e., $mathbb{1}mathbb{1}^T$ where the largest eigenvalue is $n$ and rest of the eigenvalues are zero.
In this example, the largest eigenvalue increases from $1$ to $n$ when correlation matrix changes from the identity matrix to the rank-1 matrix. On the other hand, rest of the eigenvalues decreases from $1$ to $0$.
In order to prove the above mentioned claim, will it be sufficient:
if we can show that the largest eigenvalue path from the identity matrix to a matrix of ones, i.e., $mathbb{1}mathbb{1}^T$ is monotonically non-decreasing. Here, we will change only off-diagonal elements which always lies between -1 to 1. We also establish similar behaviour for rest of the eigenvalues ?
Can you comment on this approach? If you think, it could be a right direction. Do you have any suggestion how should I start to prove/disprove the claim?
linear-algebra matrices eigenvalues-eigenvectors linear-transformations perturbation-theory
linear-algebra matrices eigenvalues-eigenvectors linear-transformations perturbation-theory
asked Dec 9 '18 at 8:38
harihari
63
asked Dec 9 '18 at 8:38
harihari
63
edited Dec 10 '18 at 1:50
hari
asked Dec 9 '18 at 8:38
harihari
63
asked Dec 9 '18 at 8:38
harihari
63
asked Dec 9 '18 at 8:38
harihari
63
63
$begingroup$
It is not clear to me if your allowed perturbation preserves the $a_i/|a_i|$ condition. If it does, that you know that the trace is still $n$ under the perturbation and hence the sum of eigenvalues is unchanged. In the 2d example that you presented, How do you know that $delta>0$? A negative $delta$ would invalidate your claim.
$endgroup$
– user617446
Dec 9 '18 at 12:53
$begingroup$
@user617446 Yes, the allowed perturbation preserves the $a_i/|a_i|$ condition. This is always ensured. You are right, in this condition, sum of eigenvalues is unchanged and somehow I should utilize this information to prove my arguments. In the 2nd example, $delta$ is positive by construction. This signify that the correlation among all columns of $A$ are increasing.
$endgroup$
– hari
Dec 9 '18 at 15:32
add a comment |
$begingroup$
It is not clear to me if your allowed perturbation preserves the $a_i/|a_i|$ condition. If it does, that you know that the trace is still $n$ under the perturbation and hence the sum of eigenvalues is unchanged. In the 2d example that you presented, How do you know that $delta>0$? A negative $delta$ would invalidate your claim.
$endgroup$
– user617446
Dec 9 '18 at 12:53
$begingroup$
@user617446 Yes, the allowed perturbation preserves the $a_i/|a_i|$ condition. This is always ensured. You are right, in this condition, sum of eigenvalues is unchanged and somehow I should utilize this information to prove my arguments. In the 2nd example, $delta$ is positive by construction. This signify that the correlation among all columns of $A$ are increasing.
$endgroup$
– hari
Dec 9 '18 at 15:32
$begingroup$
It is not clear to me if your allowed perturbation preserves the $a_i/|a_i|$ condition. If it does, that you know that the trace is still $n$ under the perturbation and hence the sum of eigenvalues is unchanged. In the 2d example that you presented, How do you know that $delta>0$? A negative $delta$ would invalidate your claim.
$endgroup$
– user617446
Dec 9 '18 at 12:53
$begingroup$
It is not clear to me if your allowed perturbation preserves the $a_i/|a_i|$ condition. If it does, that you know that the trace is still $n$ under the perturbation and hence the sum of eigenvalues is unchanged. In the 2d example that you presented, How do you know that $delta>0$? A negative $delta$ would invalidate your claim.
$endgroup$
– user617446
Dec 9 '18 at 12:53
$begingroup$
@user617446 Yes, the allowed perturbation preserves the $a_i/|a_i|$ condition. This is always ensured. You are right, in this condition, sum of eigenvalues is unchanged and somehow I should utilize this information to prove my arguments. In the 2nd example, $delta$ is positive by construction. This signify that the correlation among all columns of $A$ are increasing.
$endgroup$
– hari
Dec 9 '18 at 15:32
$begingroup$
@user617446 Yes, the allowed perturbation preserves the $a_i/|a_i|$ condition. This is always ensured. You are right, in this condition, sum of eigenvalues is unchanged and somehow I should utilize this information to prove my arguments. In the 2nd example, $delta$ is positive by construction. This signify that the correlation among all columns of $A$ are increasing.
$endgroup$
– hari
Dec 9 '18 at 15:32
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3032161%2fchange-in-eigenvalues-due-to-perturbation-to-a-correlation-matrix%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3032161%2fchange-in-eigenvalues-due-to-perturbation-to-a-correlation-matrix%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
It is not clear to me if your allowed perturbation preserves the $a_i/|a_i|$ condition. If it does, that you know that the trace is still $n$ under the perturbation and hence the sum of eigenvalues is unchanged. In the 2d example that you presented, How do you know that $delta>0$? A negative $delta$ would invalidate your claim.
$endgroup$
– user617446
Dec 9 '18 at 12:53
$begingroup$
@user617446 Yes, the allowed perturbation preserves the $a_i/|a_i|$ condition. This is always ensured. You are right, in this condition, sum of eigenvalues is unchanged and somehow I should utilize this information to prove my arguments. In the 2nd example, $delta$ is positive by construction. This signify that the correlation among all columns of $A$ are increasing.
$endgroup$
– hari
Dec 9 '18 at 15:32