An Inconsistency in Numerical Approximation











up vote
6
down vote

favorite












Consider the expression



$$
10^5 - frac{10^{10}}{1+10^5}.
$$



Using the elementary properties of fractions we can evaluate the expression as



$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$



Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get



$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$



The same logic works for



$$
10^p - frac{10^{2p}}{1+10^p}
$$



for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.



Is there an easy explanation of what's going on here?










share|cite|improve this question




















  • 4




    en.wikipedia.org/wiki/Loss_of_significance might be a starting point
    – Thomas
    Nov 28 at 19:41






  • 5




    This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
    – kimchi lover
    Nov 28 at 19:42










  • relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
    – Vasya
    Nov 28 at 19:48










  • @Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
    – SZN
    Nov 28 at 19:50






  • 1




    Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
    – Servaes
    Nov 29 at 1:17

















up vote
6
down vote

favorite












Consider the expression



$$
10^5 - frac{10^{10}}{1+10^5}.
$$



Using the elementary properties of fractions we can evaluate the expression as



$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$



Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get



$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$



The same logic works for



$$
10^p - frac{10^{2p}}{1+10^p}
$$



for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.



Is there an easy explanation of what's going on here?










share|cite|improve this question




















  • 4




    en.wikipedia.org/wiki/Loss_of_significance might be a starting point
    – Thomas
    Nov 28 at 19:41






  • 5




    This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
    – kimchi lover
    Nov 28 at 19:42










  • relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
    – Vasya
    Nov 28 at 19:48










  • @Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
    – SZN
    Nov 28 at 19:50






  • 1




    Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
    – Servaes
    Nov 29 at 1:17















up vote
6
down vote

favorite









up vote
6
down vote

favorite











Consider the expression



$$
10^5 - frac{10^{10}}{1+10^5}.
$$



Using the elementary properties of fractions we can evaluate the expression as



$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$



Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get



$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$



The same logic works for



$$
10^p - frac{10^{2p}}{1+10^p}
$$



for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.



Is there an easy explanation of what's going on here?










share|cite|improve this question















Consider the expression



$$
10^5 - frac{10^{10}}{1+10^5}.
$$



Using the elementary properties of fractions we can evaluate the expression as



$$
10^5 - frac{10^{10}}{1+10^5} = frac{10^5 + 10^{10} - 10^{10}}{1+10^5} = frac{10^5}{1+10^5}approx 1.
$$



Note that the approximation $10^5+1 approx 10^5$ is used in the last step. Now suppose we use the same approximation, but apply it before we perform the subtraction. We get



$$
10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} = 0.
$$



The same logic works for



$$
10^p - frac{10^{2p}}{1+10^p}
$$



for arbitrary large $p$, so it cannot be simply an issue with the accuracy of the approximation.



Is there an easy explanation of what's going on here?







arithmetic approximation fractions






share|cite|improve this question















share|cite|improve this question













share|cite|improve this question




share|cite|improve this question








edited Nov 29 at 11:14









Servaes

22.2k33793




22.2k33793










asked Nov 28 at 19:37









SZN

2,713720




2,713720








  • 4




    en.wikipedia.org/wiki/Loss_of_significance might be a starting point
    – Thomas
    Nov 28 at 19:41






  • 5




    This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
    – kimchi lover
    Nov 28 at 19:42










  • relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
    – Vasya
    Nov 28 at 19:48










  • @Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
    – SZN
    Nov 28 at 19:50






  • 1




    Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
    – Servaes
    Nov 29 at 1:17
















  • 4




    en.wikipedia.org/wiki/Loss_of_significance might be a starting point
    – Thomas
    Nov 28 at 19:41






  • 5




    This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
    – kimchi lover
    Nov 28 at 19:42










  • relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
    – Vasya
    Nov 28 at 19:48










  • @Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
    – SZN
    Nov 28 at 19:50






  • 1




    Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
    – Servaes
    Nov 29 at 1:17










4




4




en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
Nov 28 at 19:41




en.wikipedia.org/wiki/Loss_of_significance might be a starting point
– Thomas
Nov 28 at 19:41




5




5




This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
Nov 28 at 19:42




This is called "catastrophic cancellation"; see en.wikipedia.org/wiki/Loss_of_significance . The subject of numerical analysis is largely devoted to studying & combating this phenomenon, teaching in general how to calculate according to your first example.
– kimchi lover
Nov 28 at 19:42












relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
Nov 28 at 19:48




relative to the numbers involved, the error is still pretty small, $(1-0)/10^5=10^{-5}$
– Vasya
Nov 28 at 19:48












@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
Nov 28 at 19:50




@Vasya yes, but the difference between $1$ and $0$ leads to considerably different answers if this factor happens to be multiplying another and the approximation is applied improperly!
– SZN
Nov 28 at 19:50




1




1




Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
Nov 29 at 1:17






Interestingly, the error compounds quickly; $$10^5 - frac{10^{10}}{1+10^5} approx 10^5 - frac{10^{10}}{10^5} approx 10^5 - frac{10^{10}}{10^5-1}approxldots approx 10^5 - frac{10^{10}}{2} approx 10^5 - frac{10^{10}}{1} approx-10^{10}.$$
– Servaes
Nov 29 at 1:17












3 Answers
3






active

oldest

votes

















up vote
6
down vote



accepted










It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
$$ x - frac{x^2}{1+x}$$



Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
so that
$$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$



In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.






share|cite|improve this answer





















  • Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
    – SZN
    Nov 28 at 20:03




















up vote
1
down vote













The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.






share|cite|improve this answer




























    up vote
    1
    down vote













    There is no paradox.



    When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.



    But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.






    share|cite|improve this answer























    • I don't recall claiming there was a paradox, only a numerical issue.
      – SZN
      Nov 28 at 20:05











    Your Answer





    StackExchange.ifUsing("editor", function () {
    return StackExchange.using("mathjaxEditing", function () {
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    });
    });
    }, "mathjax-editing");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "69"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    noCode: true, onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3017585%2fan-inconsistency-in-numerical-approximation%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    3 Answers
    3






    active

    oldest

    votes








    3 Answers
    3






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    6
    down vote



    accepted










    It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
    $$ x - frac{x^2}{1+x}$$



    Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
    so that
    $$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$



    In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.






    share|cite|improve this answer





















    • Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
      – SZN
      Nov 28 at 20:03

















    up vote
    6
    down vote



    accepted










    It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
    $$ x - frac{x^2}{1+x}$$



    Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
    so that
    $$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$



    In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.






    share|cite|improve this answer





















    • Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
      – SZN
      Nov 28 at 20:03















    up vote
    6
    down vote



    accepted







    up vote
    6
    down vote



    accepted






    It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
    $$ x - frac{x^2}{1+x}$$



    Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
    so that
    $$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$



    In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.






    share|cite|improve this answer












    It is simply an issue of accuracy of approximation. Let me write $x = 10^p$. Then your expression is
    $$ x - frac{x^2}{1+x}$$



    Note that $$frac{x^2}{1+x} = frac{x}{1/x + 1} = x (1 - 1/x + O(1/x^2)) = x - 1 + O(1/x)$$
    so that
    $$ x - frac{x^2}{1+x} = x - (x - 1 + O(1/x)) = 1 + O(1/x)$$



    In your second calculation you only evaluated $x^2/(1+x)$ to within $O(1)$, not $O(1/x)$, so naturally you have an error at the end that is $O(1)$.







    share|cite|improve this answer












    share|cite|improve this answer



    share|cite|improve this answer










    answered Nov 28 at 19:50









    Robert Israel

    317k23206457




    317k23206457












    • Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
      – SZN
      Nov 28 at 20:03




















    • Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
      – SZN
      Nov 28 at 20:03


















    Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
    – SZN
    Nov 28 at 20:03






    Thank you! This makes a lot of sense. I should have been clearer with my use of the word "approximation", which I took to be the magnitude of the error $$e = 1-frac{10^p}{1+10^{p}}$$ rather than the order to which this approximation effectively takes the Taylor series.
    – SZN
    Nov 28 at 20:03












    up vote
    1
    down vote













    The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.






    share|cite|improve this answer

























      up vote
      1
      down vote













      The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.






      share|cite|improve this answer























        up vote
        1
        down vote










        up vote
        1
        down vote









        The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.






        share|cite|improve this answer












        The first approximation is fine. The second on is not, because, $10^5$ and $dfrac{10^{10}}{1+10^5}$ are large numbers with approximately the same size. You are saying that since $10,001$ is close to $10,000$, then $1$ is close to $0$.







        share|cite|improve this answer












        share|cite|improve this answer



        share|cite|improve this answer










        answered Nov 28 at 19:43









        José Carlos Santos

        147k22117217




        147k22117217






















            up vote
            1
            down vote













            There is no paradox.



            When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.



            But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.






            share|cite|improve this answer























            • I don't recall claiming there was a paradox, only a numerical issue.
              – SZN
              Nov 28 at 20:05















            up vote
            1
            down vote













            There is no paradox.



            When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.



            But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.






            share|cite|improve this answer























            • I don't recall claiming there was a paradox, only a numerical issue.
              – SZN
              Nov 28 at 20:05













            up vote
            1
            down vote










            up vote
            1
            down vote









            There is no paradox.



            When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.



            But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.






            share|cite|improve this answer














            There is no paradox.



            When you approximate $$frac{10^5}{1+10^5}=1-0.000099999000cdots$$ with $1$, the error is on the order of $10^{-5}$.



            But in the second case, the same error is multiplied by $10^5$, so that it is no more negligible.







            share|cite|improve this answer














            share|cite|improve this answer



            share|cite|improve this answer








            edited Nov 28 at 20:04

























            answered Nov 28 at 19:57









            Yves Daoust

            123k668219




            123k668219












            • I don't recall claiming there was a paradox, only a numerical issue.
              – SZN
              Nov 28 at 20:05


















            • I don't recall claiming there was a paradox, only a numerical issue.
              – SZN
              Nov 28 at 20:05
















            I don't recall claiming there was a paradox, only a numerical issue.
            – SZN
            Nov 28 at 20:05




            I don't recall claiming there was a paradox, only a numerical issue.
            – SZN
            Nov 28 at 20:05


















            draft saved

            draft discarded




















































            Thanks for contributing an answer to Mathematics Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3017585%2fan-inconsistency-in-numerical-approximation%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Plaza Victoria

            Puebla de Zaragoza

            Musa