This is my reasoning also. I wear a button up shirt nearly every day. And always button starting from 2nd to the top and go down from there. Buttoning from the bottom up seems totally unthinkable.