Morality WAS a significant factor. Like we can't look at the radical republicans, the barnburners and the free soilers and pretend there wasn't a significant amount of people who were morally outraged by slavery. But of course this morality was in part informed by economic interest. The American political structure in much of the country was built around paving over issues of class and economic relations by providing "free" land and thus an incredible amount of perceived social mobility, which created a large class of small holding farmers and petite bourgeoise. These people's economics interests were not aligned with slavery, because they were in direct competition with slave staffed farms and the cash crop plantations of the south.

So as the country settled more and more western territory, these small holders were interested in limited the expansion of slavery, and the slave holding classes of the south were interested in expanding it. This came to violent clashes, which intensified the opposition to slavery, because not only are slave plantations a direct competition to the ideal of the American settler-farmer, he's also hiring dudes to go around murdering the American settler-farmer for it. So whenever America expanded there was intense conflict over whether the new territory should be a slave territory, and therefore adopt the plantation based economic model, or align itself with the northern faction. This was important not only because of the individual economic and political interests of the people settling the land, but new states and territories meant new representatives and senators who would affect the balance of the legislature. The South had a totally outsized level of political representation, but they were intensely paranoid of letting that slip, and many people in other states were pissed that like seven guys in slave states had as much voting power as entire towns and the southern senators were lording that shit over everyone. So on top of everything else, "The south" is paranoid that new states won't be slave states so they don't get to hold all the power, "the north" wants new states to not be slaves states because southern senators are the fucking worst. That's why the south secedes despite no one actually coming for their slaves yet, they're worried they eventually won't hold all the power and lose their control of the senate. (I'm using "The south" and "The north" as short hands, things didn't neatly align this way. Also I'm totally ignoring the existence of factions like the doughfaces, the worst people in the world)