Ballet and the Alexander Technique are helpful.
Working on your posture and becoming more aware of what you are doing with your body is the only way to fix this. Try to remember that in order for your weight to be in the right place over your skate, you need to have everything in alignment, from your head through shoulders, hips, and ankle.