Be sure to install a brand new needle.

If the needle bar is moving to the side when set on a straight-stitch, there is probably something loose or bent in the machine. Would recommend having it checked by a technician. Could also be the cause of the thread clumping or could be something else going on.

...