An Inconsistency in Numerical Approximation
up vote
6
down vote
favorite
Consider the expression
$$
10^5 - frac{10^{10}}{1+10^5}.
$$
Using the elementary properties of fractions we can evaluate the expression as
$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$
Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get
$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$
The same logic works for
$$
10^p - frac{10^{2p}}{1+10^p}
$$
for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.
Is there an easy explanation of what's going on here?
arithmetic approximation fractions
|
show 3 more comments
up vote
6
down vote
favorite
Consider the expression
$$
10^5 - frac{10^{10}}{1+10^5}.
$$
Using the elementary properties of fractions we can evaluate the expression as
$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$
Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get
$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$
The same logic works for
$$
10^p - frac{10^{2p}}{1+10^p}
$$
for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.
Is there an easy explanation of what's going on here?
arithmetic approximation fractions
4
en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
Nov 28 at 19:41
5
This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
Nov 28 at 19:42
relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
Nov 28 at 19:48
@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
Nov 28 at 19:50
1
Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
Nov 29 at 1:17
|
show 3 more comments
up vote
6
down vote
favorite
up vote
6
down vote
favorite
Consider the expression
$$
10^5 - frac{10^{10}}{1+10^5}.
$$
Using the elementary properties of fractions we can evaluate the expression as
$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$
Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get
$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$
The same logic works for
$$
10^p - frac{10^{2p}}{1+10^p}
$$
for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.
Is there an easy explanation of what's going on here?
arithmetic approximation fractions
Consider the expression
$$
10^5 - frac{10^{10}}{1+10^5}.
$$
Using the elementary properties of fractions we can evaluate the expression as
$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$
Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get
$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$
The same logic works for
$$
10^p - frac{10^{2p}}{1+10^p}
$$
for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.
Is there an easy explanation of what's going on here?
arithmetic approximation fractions
arithmetic approximation fractions
edited Nov 29 at 11:14
Servaes
22.2k33793
22.2k33793
asked Nov 28 at 19:37
SZN
2,713720
2,713720
4
en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
Nov 28 at 19:41
5
This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
Nov 28 at 19:42
relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
Nov 28 at 19:48
@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
Nov 28 at 19:50
1
Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
Nov 29 at 1:17
|
show 3 more comments
4
en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
Nov 28 at 19:41
5
This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
Nov 28 at 19:42
relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
Nov 28 at 19:48
@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
Nov 28 at 19:50
1
Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
Nov 29 at 1:17
4
4
en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
Nov 28 at 19:41
en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
Nov 28 at 19:41
5
5
This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
Nov 28 at 19:42
This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
Nov 28 at 19:42
relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
Nov 28 at 19:48
relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
Nov 28 at 19:48
@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
Nov 28 at 19:50
@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
Nov 28 at 19:50
1
1
Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
Nov 29 at 1:17
Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
Nov 29 at 1:17
|
show 3 more comments
3 Answers
3
active
oldest
votes
up vote
6
down vote
accepted
It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
$$ x - frac{x^2}{1+x}$$
Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
so that
$$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$
In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
Nov 28 at 20:03
add a comment |
up vote
1
down vote
The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.
add a comment |
up vote
1
down vote
There is no paradox.
When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.
But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
Nov 28 at 20:05
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3017585%2fan-inconsistency-in-numerical-approximation%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
6
down vote
accepted
It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
$$ x - frac{x^2}{1+x}$$
Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
so that
$$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$
In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
Nov 28 at 20:03
add a comment |
up vote
6
down vote
accepted
It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
$$ x - frac{x^2}{1+x}$$
Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
so that
$$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$
In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
Nov 28 at 20:03
add a comment |
up vote
6
down vote
accepted
up vote
6
down vote
accepted
It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
$$ x - frac{x^2}{1+x}$$
Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
so that
$$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$
In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.
It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
$$ x - frac{x^2}{1+x}$$
Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
so that
$$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$
In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.
answered Nov 28 at 19:50
Robert Israel
317k23206457
317k23206457
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
Nov 28 at 20:03
add a comment |
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
Nov 28 at 20:03
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
Nov 28 at 20:03
Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
– SZN
Nov 28 at 20:03
add a comment |
up vote
1
down vote
The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.
add a comment |
up vote
1
down vote
The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.
add a comment |
up vote
1
down vote
up vote
1
down vote
The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.
The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.
answered Nov 28 at 19:43
José Carlos Santos
147k22117217
147k22117217
add a comment |
add a comment |
up vote
1
down vote
There is no paradox.
When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.
But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
Nov 28 at 20:05
add a comment |
up vote
1
down vote
There is no paradox.
When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.
But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
Nov 28 at 20:05
add a comment |
up vote
1
down vote
up vote
1
down vote
There is no paradox.
When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.
But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.
There is no paradox.
When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.
But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.
edited Nov 28 at 20:04
answered Nov 28 at 19:57
Yves Daoust
123k668219
123k668219
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
Nov 28 at 20:05
add a comment |
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
Nov 28 at 20:05
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
Nov 28 at 20:05
I don't recall claiming there was a paradox, only a numerical issue.
– SZN
Nov 28 at 20:05
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3017585%2fan-inconsistency-in-numerical-approximation%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
4
en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
Nov 28 at 19:41
5
This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
Nov 28 at 19:42
relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
Nov 28 at 19:48
@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
Nov 28 at 19:50
1
Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
Nov 29 at 1:17