How to infer difference of population proportion between two groups when proportion is small?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
$begingroup$
I have a dataset where the issue is of this form.
There are two groups, Group A (N=5000) and Group B (N=1000). Let's say 5 people in Group A develop a certain disease, and only 2 in group B do.
Then the proportion for A is 5/5000 -> 0.001 and for B it is 2/1000 -> 0.002.
How can I test if the proportion between these two groups is statistically significant?
The tests I found online rely on the Central Limit Theorem, such that np>=10 and n(1-p) >= 10, which does not hold for my dataset. Are there any other approaches?
inference proportion
asked Apr 21 at 22:52
maxmax
1133
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I have a dataset where the issue is of this form.
There are two groups, Group A (N=5000) and Group B (N=1000). Let's say 5 people in Group A develop a certain disease, and only 2 in group B do.
Then the proportion for A is 5/5000 -> 0.001 and for B it is 2/1000 -> 0.002.
How can I test if the proportion between these two groups is statistically significant?
The tests I found online rely on the Central Limit Theorem, such that np>=10 and n(1-p) >= 10, which does not hold for my dataset. Are there any other approaches?
inference proportion
asked Apr 21 at 22:52
maxmax
1133
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
$begingroup$
Use Fisher Exact Test per discussion in Answer.
$endgroup$
– BruceET
Apr 22 at 2:38
add a comment |
$begingroup$
I have a dataset where the issue is of this form.
There are two groups, Group A (N=5000) and Group B (N=1000). Let's say 5 people in Group A develop a certain disease, and only 2 in group B do.
Then the proportion for A is 5/5000 -> 0.001 and for B it is 2/1000 -> 0.002.
How can I test if the proportion between these two groups is statistically significant?
The tests I found online rely on the Central Limit Theorem, such that np>=10 and n(1-p) >= 10, which does not hold for my dataset. Are there any other approaches?
inference proportion
asked Apr 21 at 22:52
maxmax
1133
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I have a dataset where the issue is of this form.
There are two groups, Group A (N=5000) and Group B (N=1000). Let's say 5 people in Group A develop a certain disease, and only 2 in group B do.
Then the proportion for A is 5/5000 -> 0.001 and for B it is 2/1000 -> 0.002.
How can I test if the proportion between these two groups is statistically significant?
The tests I found online rely on the Central Limit Theorem, such that np>=10 and n(1-p) >= 10, which does not hold for my dataset. Are there any other approaches?
inference proportion
inference proportion
asked Apr 21 at 22:52
maxmax
1133
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Apr 21 at 22:52
maxmax
1133
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Apr 21 at 22:52
maxmax
1133
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Apr 21 at 22:52
maxmax
1133
asked Apr 21 at 22:52
maxmax
1133
1133
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
max is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$begingroup$
Use Fisher Exact Test per discussion in Answer.
$endgroup$
– BruceET
Apr 22 at 2:38
add a comment |
$begingroup$
Use Fisher Exact Test per discussion in Answer.
$endgroup$
– BruceET
Apr 22 at 2:38
$begingroup$
Use Fisher Exact Test per discussion in Answer.
$endgroup$
– BruceET
Apr 22 at 2:38
$begingroup$
Use Fisher Exact Test per discussion in Answer.
$endgroup$
– BruceET
Apr 22 at 2:38
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
The traditional approach would be to do a 2-sample test for a difference in proportions: In Minitab, results of this test are as shown below. The warning about the first P-value from a normal approximation causes doubt,
roughly for the reasons you mention.
However, the result from Fisher's exact test uses an exact hypergeometric
probability. It also shows no significant difference.
Test and CI for Two Proportions
Sample X N Sample p
1 5 5000 0.001000
2 2 1000 0.002000
Difference = p (1) - p (2)
Estimate for difference: -0.001
95% upper bound for difference: 0.00143738
Test for difference = 0 (vs < 0):
Z = -0.67 P-Value = 0.250
* NOTE * The normal approximation may be
inaccurate for small samples.
Fisher’s exact test: P-Value = 0.330
A direct hypergeometric computation in R can be argued
as follows. Suppose an urn contains $5000$ tokens marked A and $1000$ marked B. Seven tokens are taken
at random without replacement, corresponding to disease.
What is the probability five or fewer of those are marked A?
The answer is
$$sum_{k=0}^5frac{{5000 choose k}{1000 choose 7-k}}{{6000 choose 7}} = 0.3302,$$
which agrees with the P-value from Fisher's exact test.
In R, the computation can be done in terms of a hypergeometric CDF:
phyper(5, 5000, 1000, 7)
[1] 0.330204
Here is a plot of this hypergeometric distribution. The P-value is the sum of the heights of the bars to the left of the vertical dotted line.
answered Apr 22 at 2:07
BruceETBruceET
7,2861721
$endgroup$
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "65"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
max is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f404331%2fhow-to-infer-difference-of-population-proportion-between-two-groups-when-proport%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
The traditional approach would be to do a 2-sample test for a difference in proportions: In Minitab, results of this test are as shown below. The warning about the first P-value from a normal approximation causes doubt,
roughly for the reasons you mention.
However, the result from Fisher's exact test uses an exact hypergeometric
probability. It also shows no significant difference.
Test and CI for Two Proportions
Sample X N Sample p
1 5 5000 0.001000
2 2 1000 0.002000
Difference = p (1) - p (2)
Estimate for difference: -0.001
95% upper bound for difference: 0.00143738
Test for difference = 0 (vs < 0):
Z = -0.67 P-Value = 0.250
* NOTE * The normal approximation may be
inaccurate for small samples.
Fisher’s exact test: P-Value = 0.330
A direct hypergeometric computation in R can be argued
as follows. Suppose an urn contains $5000$ tokens marked A and $1000$ marked B. Seven tokens are taken
at random without replacement, corresponding to disease.
What is the probability five or fewer of those are marked A?
The answer is
$$sum_{k=0}^5frac{{5000 choose k}{1000 choose 7-k}}{{6000 choose 7}} = 0.3302,$$
which agrees with the P-value from Fisher's exact test.
In R, the computation can be done in terms of a hypergeometric CDF:
phyper(5, 5000, 1000, 7)
[1] 0.330204
Here is a plot of this hypergeometric distribution. The P-value is the sum of the heights of the bars to the left of the vertical dotted line.
answered Apr 22 at 2:07
BruceETBruceET
7,2861721
$endgroup$
add a comment |
$begingroup$
The traditional approach would be to do a 2-sample test for a difference in proportions: In Minitab, results of this test are as shown below. The warning about the first P-value from a normal approximation causes doubt,
roughly for the reasons you mention.
However, the result from Fisher's exact test uses an exact hypergeometric
probability. It also shows no significant difference.
Test and CI for Two Proportions
Sample X N Sample p
1 5 5000 0.001000
2 2 1000 0.002000
Difference = p (1) - p (2)
Estimate for difference: -0.001
95% upper bound for difference: 0.00143738
Test for difference = 0 (vs < 0):
Z = -0.67 P-Value = 0.250
* NOTE * The normal approximation may be
inaccurate for small samples.
Fisher’s exact test: P-Value = 0.330
A direct hypergeometric computation in R can be argued
as follows. Suppose an urn contains $5000$ tokens marked A and $1000$ marked B. Seven tokens are taken
at random without replacement, corresponding to disease.
What is the probability five or fewer of those are marked A?
The answer is
$$sum_{k=0}^5frac{{5000 choose k}{1000 choose 7-k}}{{6000 choose 7}} = 0.3302,$$
which agrees with the P-value from Fisher's exact test.
In R, the computation can be done in terms of a hypergeometric CDF:
phyper(5, 5000, 1000, 7)
[1] 0.330204
Here is a plot of this hypergeometric distribution. The P-value is the sum of the heights of the bars to the left of the vertical dotted line.
answered Apr 22 at 2:07
BruceETBruceET
7,2861721
$endgroup$
add a comment |
$begingroup$
The traditional approach would be to do a 2-sample test for a difference in proportions: In Minitab, results of this test are as shown below. The warning about the first P-value from a normal approximation causes doubt,
roughly for the reasons you mention.
However, the result from Fisher's exact test uses an exact hypergeometric
probability. It also shows no significant difference.
Test and CI for Two Proportions
Sample X N Sample p
1 5 5000 0.001000
2 2 1000 0.002000
Difference = p (1) - p (2)
Estimate for difference: -0.001
95% upper bound for difference: 0.00143738
Test for difference = 0 (vs < 0):
Z = -0.67 P-Value = 0.250
* NOTE * The normal approximation may be
inaccurate for small samples.
Fisher’s exact test: P-Value = 0.330
A direct hypergeometric computation in R can be argued
as follows. Suppose an urn contains $5000$ tokens marked A and $1000$ marked B. Seven tokens are taken
at random without replacement, corresponding to disease.
What is the probability five or fewer of those are marked A?
The answer is
$$sum_{k=0}^5frac{{5000 choose k}{1000 choose 7-k}}{{6000 choose 7}} = 0.3302,$$
which agrees with the P-value from Fisher's exact test.
In R, the computation can be done in terms of a hypergeometric CDF:
phyper(5, 5000, 1000, 7)
[1] 0.330204
Here is a plot of this hypergeometric distribution. The P-value is the sum of the heights of the bars to the left of the vertical dotted line.
answered Apr 22 at 2:07
BruceETBruceET
7,2861721
$endgroup$
The traditional approach would be to do a 2-sample test for a difference in proportions: In Minitab, results of this test are as shown below. The warning about the first P-value from a normal approximation causes doubt,
roughly for the reasons you mention.
However, the result from Fisher's exact test uses an exact hypergeometric
probability. It also shows no significant difference.
Test and CI for Two Proportions
Sample X N Sample p
1 5 5000 0.001000
2 2 1000 0.002000
Difference = p (1) - p (2)
Estimate for difference: -0.001
95% upper bound for difference: 0.00143738
Test for difference = 0 (vs < 0):
Z = -0.67 P-Value = 0.250
* NOTE * The normal approximation may be
inaccurate for small samples.
Fisher’s exact test: P-Value = 0.330
A direct hypergeometric computation in R can be argued
as follows. Suppose an urn contains $5000$ tokens marked A and $1000$ marked B. Seven tokens are taken
at random without replacement, corresponding to disease.
What is the probability five or fewer of those are marked A?
The answer is
$$sum_{k=0}^5frac{{5000 choose k}{1000 choose 7-k}}{{6000 choose 7}} = 0.3302,$$
which agrees with the P-value from Fisher's exact test.
In R, the computation can be done in terms of a hypergeometric CDF:
phyper(5, 5000, 1000, 7)
[1] 0.330204
Here is a plot of this hypergeometric distribution. The P-value is the sum of the heights of the bars to the left of the vertical dotted line.
answered Apr 22 at 2:07
BruceETBruceET
7,2861721
answered Apr 22 at 2:07
BruceETBruceET
7,2861721
answered Apr 22 at 2:07
BruceETBruceET
7,2861721
answered Apr 22 at 2:07
BruceETBruceET
7,2861721
7,2861721
add a comment |
add a comment |
max is a new contributor. Be nice, and check out our Code of Conduct.
max is a new contributor. Be nice, and check out our Code of Conduct.
max is a new contributor. Be nice, and check out our Code of Conduct.
max is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f404331%2fhow-to-infer-difference-of-population-proportion-between-two-groups-when-proport%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Use Fisher Exact Test per discussion in Answer.
$endgroup$
– BruceET
Apr 22 at 2:38