To me, single mother/father isn't negative. It's simply a way to explain their situation; that they are raising a child by themselves and for whatever reason they don't have a partner.
Honestly the first time I've heard someone think of it in a negative way. Mostly I hear people say single mother/father with sincere respect as they know raising a kid is difficult, and can't imagine doing it by themselves.
*Quick edit to affirm the fact that single fathers are relatively rare as we live in a patriarchy.