Read the other replies but this is what clicked it for me:
Between step 2 and 3, you applied the derivative to all of the x's in the sum (x+x+x...) but ignored the x in the "x times".
This nonstandard notation helps to hide that. If you wrote this in sigma notation, you'd have:
If you differentiate this with respect to x, you can't ignore the x in the sigma limit. When differentiating a summation where the limits are a function of the target variable I believe you need to use Leibniz rule(?), but I'll leave it there