"The day starts at night" sounds silly because it seems to be a contradiction. But really, how else could it be?

Either, day starts at day ... but then it was already day. Or, day starts at night ... unless we come up with additional entities like dusk or dawn.

And since we haven't introduced them yet, day has to start at night, as a necessity.

Of course the actual silly thing is that it's still night right after day has started.