Line Of Best Fit With Perpendicular Error












1












$begingroup$


The standard statistical formula for the least squares error gives us a line that minimises the sum of the vertical distances of the sample points to the line. Suppose that I wanted to find the equation of a line that minimises the sum of the perpendicular distance of the points to the line, is there a way of analytically solving this problem?










share|cite|improve this question









$endgroup$












  • $begingroup$
    Make a higher dimensional level set instead.
    $endgroup$
    – mathreadler
    Dec 21 '18 at 22:01
















1












$begingroup$


The standard statistical formula for the least squares error gives us a line that minimises the sum of the vertical distances of the sample points to the line. Suppose that I wanted to find the equation of a line that minimises the sum of the perpendicular distance of the points to the line, is there a way of analytically solving this problem?










share|cite|improve this question









$endgroup$












  • $begingroup$
    Make a higher dimensional level set instead.
    $endgroup$
    – mathreadler
    Dec 21 '18 at 22:01














1












1








1





$begingroup$


The standard statistical formula for the least squares error gives us a line that minimises the sum of the vertical distances of the sample points to the line. Suppose that I wanted to find the equation of a line that minimises the sum of the perpendicular distance of the points to the line, is there a way of analytically solving this problem?










share|cite|improve this question









$endgroup$




The standard statistical formula for the least squares error gives us a line that minimises the sum of the vertical distances of the sample points to the line. Suppose that I wanted to find the equation of a line that minimises the sum of the perpendicular distance of the points to the line, is there a way of analytically solving this problem?







geometry statistics regression






share|cite|improve this question













share|cite|improve this question











share|cite|improve this question




share|cite|improve this question










asked Dec 20 '18 at 10:17









Elie BergmanElie Bergman

1,998815




1,998815












  • $begingroup$
    Make a higher dimensional level set instead.
    $endgroup$
    – mathreadler
    Dec 21 '18 at 22:01


















  • $begingroup$
    Make a higher dimensional level set instead.
    $endgroup$
    – mathreadler
    Dec 21 '18 at 22:01
















$begingroup$
Make a higher dimensional level set instead.
$endgroup$
– mathreadler
Dec 21 '18 at 22:01




$begingroup$
Make a higher dimensional level set instead.
$endgroup$
– mathreadler
Dec 21 '18 at 22:01










2 Answers
2






active

oldest

votes


















2












$begingroup$

This is well-known. See https://fr.scribd.com/doc/14819165/Regressions-coniques-quadriques-circulaire-spherique . Figure below:



enter image description here



Note : For the choice of sign, compute $sum_{k=1}^n(ax_k+b-y_k)^2$ in both cases and keep the smaller.



Also, see : http://mathworld.wolfram.com/LeastSquaresFittingPerpendicularOffsets.html



Moreover, it is of interest to look at the related Principal Component Analysis method. https://en.wikipedia.org/wiki/Principal_component_analysis



A numerical example of the principal component regression is given in page 12 of this paper : https://fr.scribd.com/doc/31477970/Regressions-et-trajectoires-3D . This example is in 3D. But is is very easy to see how proceed on the same manner in 2D. which is even simpler.



LATTER ADDITION



Numerical example with the principal component method :



enter image description here



Comparison with the above least mean square offset method :



enter image description here



The results of both methods are exactly the same, as it is analytically expected.



Graphical representation :



enter image description here






share|cite|improve this answer











$endgroup$













  • $begingroup$
    Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
    $endgroup$
    – Claude Leibovici
    Dec 22 '18 at 6:31










  • $begingroup$
    Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
    $endgroup$
    – JJacquelin
    Dec 22 '18 at 7:30










  • $begingroup$
    Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
    $endgroup$
    – Claude Leibovici
    Dec 22 '18 at 7:46












  • $begingroup$
    @Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
    $endgroup$
    – JJacquelin
    Dec 22 '18 at 8:53










  • $begingroup$
    Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
    $endgroup$
    – Claude Leibovici
    Dec 22 '18 at 8:56



















0












$begingroup$

Just build a function of higher in-dimensionality than whatever you tried to fit firstly. And instead of fitting to values you fit towards a level-set.



So instead of fitting to $$f(x)=kx+m, f(x_k) = y_k$$
We fit to:
$$f(x,y) = ax+by+c, f(x_k,y_k) = 0$$



This will automatically handle all your roubles ( if you add some tea at the start ).






share|cite|improve this answer









$endgroup$














    Your Answer





    StackExchange.ifUsing("editor", function () {
    return StackExchange.using("mathjaxEditing", function () {
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    });
    });
    }, "mathjax-editing");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "69"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    noCode: true, onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3047373%2fline-of-best-fit-with-perpendicular-error%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    2












    $begingroup$

    This is well-known. See https://fr.scribd.com/doc/14819165/Regressions-coniques-quadriques-circulaire-spherique . Figure below:



    enter image description here



    Note : For the choice of sign, compute $sum_{k=1}^n(ax_k+b-y_k)^2$ in both cases and keep the smaller.



    Also, see : http://mathworld.wolfram.com/LeastSquaresFittingPerpendicularOffsets.html



    Moreover, it is of interest to look at the related Principal Component Analysis method. https://en.wikipedia.org/wiki/Principal_component_analysis



    A numerical example of the principal component regression is given in page 12 of this paper : https://fr.scribd.com/doc/31477970/Regressions-et-trajectoires-3D . This example is in 3D. But is is very easy to see how proceed on the same manner in 2D. which is even simpler.



    LATTER ADDITION



    Numerical example with the principal component method :



    enter image description here



    Comparison with the above least mean square offset method :



    enter image description here



    The results of both methods are exactly the same, as it is analytically expected.



    Graphical representation :



    enter image description here






    share|cite|improve this answer











    $endgroup$













    • $begingroup$
      Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
      $endgroup$
      – Claude Leibovici
      Dec 22 '18 at 6:31










    • $begingroup$
      Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
      $endgroup$
      – JJacquelin
      Dec 22 '18 at 7:30










    • $begingroup$
      Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
      $endgroup$
      – Claude Leibovici
      Dec 22 '18 at 7:46












    • $begingroup$
      @Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
      $endgroup$
      – JJacquelin
      Dec 22 '18 at 8:53










    • $begingroup$
      Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
      $endgroup$
      – Claude Leibovici
      Dec 22 '18 at 8:56
















    2












    $begingroup$

    This is well-known. See https://fr.scribd.com/doc/14819165/Regressions-coniques-quadriques-circulaire-spherique . Figure below:



    enter image description here



    Note : For the choice of sign, compute $sum_{k=1}^n(ax_k+b-y_k)^2$ in both cases and keep the smaller.



    Also, see : http://mathworld.wolfram.com/LeastSquaresFittingPerpendicularOffsets.html



    Moreover, it is of interest to look at the related Principal Component Analysis method. https://en.wikipedia.org/wiki/Principal_component_analysis



    A numerical example of the principal component regression is given in page 12 of this paper : https://fr.scribd.com/doc/31477970/Regressions-et-trajectoires-3D . This example is in 3D. But is is very easy to see how proceed on the same manner in 2D. which is even simpler.



    LATTER ADDITION



    Numerical example with the principal component method :



    enter image description here



    Comparison with the above least mean square offset method :



    enter image description here



    The results of both methods are exactly the same, as it is analytically expected.



    Graphical representation :



    enter image description here






    share|cite|improve this answer











    $endgroup$













    • $begingroup$
      Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
      $endgroup$
      – Claude Leibovici
      Dec 22 '18 at 6:31










    • $begingroup$
      Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
      $endgroup$
      – JJacquelin
      Dec 22 '18 at 7:30










    • $begingroup$
      Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
      $endgroup$
      – Claude Leibovici
      Dec 22 '18 at 7:46












    • $begingroup$
      @Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
      $endgroup$
      – JJacquelin
      Dec 22 '18 at 8:53










    • $begingroup$
      Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
      $endgroup$
      – Claude Leibovici
      Dec 22 '18 at 8:56














    2












    2








    2





    $begingroup$

    This is well-known. See https://fr.scribd.com/doc/14819165/Regressions-coniques-quadriques-circulaire-spherique . Figure below:



    enter image description here



    Note : For the choice of sign, compute $sum_{k=1}^n(ax_k+b-y_k)^2$ in both cases and keep the smaller.



    Also, see : http://mathworld.wolfram.com/LeastSquaresFittingPerpendicularOffsets.html



    Moreover, it is of interest to look at the related Principal Component Analysis method. https://en.wikipedia.org/wiki/Principal_component_analysis



    A numerical example of the principal component regression is given in page 12 of this paper : https://fr.scribd.com/doc/31477970/Regressions-et-trajectoires-3D . This example is in 3D. But is is very easy to see how proceed on the same manner in 2D. which is even simpler.



    LATTER ADDITION



    Numerical example with the principal component method :



    enter image description here



    Comparison with the above least mean square offset method :



    enter image description here



    The results of both methods are exactly the same, as it is analytically expected.



    Graphical representation :



    enter image description here






    share|cite|improve this answer











    $endgroup$



    This is well-known. See https://fr.scribd.com/doc/14819165/Regressions-coniques-quadriques-circulaire-spherique . Figure below:



    enter image description here



    Note : For the choice of sign, compute $sum_{k=1}^n(ax_k+b-y_k)^2$ in both cases and keep the smaller.



    Also, see : http://mathworld.wolfram.com/LeastSquaresFittingPerpendicularOffsets.html



    Moreover, it is of interest to look at the related Principal Component Analysis method. https://en.wikipedia.org/wiki/Principal_component_analysis



    A numerical example of the principal component regression is given in page 12 of this paper : https://fr.scribd.com/doc/31477970/Regressions-et-trajectoires-3D . This example is in 3D. But is is very easy to see how proceed on the same manner in 2D. which is even simpler.



    LATTER ADDITION



    Numerical example with the principal component method :



    enter image description here



    Comparison with the above least mean square offset method :



    enter image description here



    The results of both methods are exactly the same, as it is analytically expected.



    Graphical representation :



    enter image description here







    share|cite|improve this answer














    share|cite|improve this answer



    share|cite|improve this answer








    edited Dec 22 '18 at 7:13

























    answered Dec 20 '18 at 10:38









    JJacquelinJJacquelin

    45.4k21857




    45.4k21857












    • $begingroup$
      Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
      $endgroup$
      – Claude Leibovici
      Dec 22 '18 at 6:31










    • $begingroup$
      Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
      $endgroup$
      – JJacquelin
      Dec 22 '18 at 7:30










    • $begingroup$
      Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
      $endgroup$
      – Claude Leibovici
      Dec 22 '18 at 7:46












    • $begingroup$
      @Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
      $endgroup$
      – JJacquelin
      Dec 22 '18 at 8:53










    • $begingroup$
      Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
      $endgroup$
      – Claude Leibovici
      Dec 22 '18 at 8:56


















    • $begingroup$
      Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
      $endgroup$
      – Claude Leibovici
      Dec 22 '18 at 6:31










    • $begingroup$
      Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
      $endgroup$
      – JJacquelin
      Dec 22 '18 at 7:30










    • $begingroup$
      Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
      $endgroup$
      – Claude Leibovici
      Dec 22 '18 at 7:46












    • $begingroup$
      @Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
      $endgroup$
      – JJacquelin
      Dec 22 '18 at 8:53










    • $begingroup$
      Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
      $endgroup$
      – Claude Leibovici
      Dec 22 '18 at 8:56
















    $begingroup$
    Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
    $endgroup$
    – Claude Leibovici
    Dec 22 '18 at 6:31




    $begingroup$
    Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
    $endgroup$
    – Claude Leibovici
    Dec 22 '18 at 6:31












    $begingroup$
    Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
    $endgroup$
    – JJacquelin
    Dec 22 '18 at 7:30




    $begingroup$
    Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
    $endgroup$
    – JJacquelin
    Dec 22 '18 at 7:30












    $begingroup$
    Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
    $endgroup$
    – Claude Leibovici
    Dec 22 '18 at 7:46






    $begingroup$
    Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
    $endgroup$
    – Claude Leibovici
    Dec 22 '18 at 7:46














    $begingroup$
    @Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
    $endgroup$
    – JJacquelin
    Dec 22 '18 at 8:53




    $begingroup$
    @Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
    $endgroup$
    – JJacquelin
    Dec 22 '18 at 8:53












    $begingroup$
    Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
    $endgroup$
    – Claude Leibovici
    Dec 22 '18 at 8:56




    $begingroup$
    Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
    $endgroup$
    – Claude Leibovici
    Dec 22 '18 at 8:56











    0












    $begingroup$

    Just build a function of higher in-dimensionality than whatever you tried to fit firstly. And instead of fitting to values you fit towards a level-set.



    So instead of fitting to $$f(x)=kx+m, f(x_k) = y_k$$
    We fit to:
    $$f(x,y) = ax+by+c, f(x_k,y_k) = 0$$



    This will automatically handle all your roubles ( if you add some tea at the start ).






    share|cite|improve this answer









    $endgroup$


















      0












      $begingroup$

      Just build a function of higher in-dimensionality than whatever you tried to fit firstly. And instead of fitting to values you fit towards a level-set.



      So instead of fitting to $$f(x)=kx+m, f(x_k) = y_k$$
      We fit to:
      $$f(x,y) = ax+by+c, f(x_k,y_k) = 0$$



      This will automatically handle all your roubles ( if you add some tea at the start ).






      share|cite|improve this answer









      $endgroup$
















        0












        0








        0





        $begingroup$

        Just build a function of higher in-dimensionality than whatever you tried to fit firstly. And instead of fitting to values you fit towards a level-set.



        So instead of fitting to $$f(x)=kx+m, f(x_k) = y_k$$
        We fit to:
        $$f(x,y) = ax+by+c, f(x_k,y_k) = 0$$



        This will automatically handle all your roubles ( if you add some tea at the start ).






        share|cite|improve this answer









        $endgroup$



        Just build a function of higher in-dimensionality than whatever you tried to fit firstly. And instead of fitting to values you fit towards a level-set.



        So instead of fitting to $$f(x)=kx+m, f(x_k) = y_k$$
        We fit to:
        $$f(x,y) = ax+by+c, f(x_k,y_k) = 0$$



        This will automatically handle all your roubles ( if you add some tea at the start ).







        share|cite|improve this answer












        share|cite|improve this answer



        share|cite|improve this answer










        answered Dec 21 '18 at 22:33









        mathreadlermathreadler

        15.4k72263




        15.4k72263






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Mathematics Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3047373%2fline-of-best-fit-with-perpendicular-error%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Plaza Victoria

            Puebla de Zaragoza

            Musa