It might not sound it, but staying stock still on skates is quite an advanced skill. They are designed to make you go, and the blades are curved which naturally leads to going on a curve.
I cannot recall exactly how I set myself to stay still as I've been doing it so long I don't think about it any more, though unlike Leif I have a feeling I am more on a slight outside edge on both skates, and I suspect feet just slightly apart and more or less parallel, weight over the middle of your feet. What I think would be rewarding for you is to experiment with different positions and try to feel where your weight is and what effect that has on the blades and on where your body wants to go.
I didn't quite follow the bit about going in circles while balancing on different parts of the blades - do you mean on one foot or both? Moving or still? Balancing on one skate and standing still is really hard and not something that beginners normally practice, though it could be helpful in developing a feel for the ice. Most skating, at least figure skating, involves being on an edge and going on a curve, though obviously one ought to be able to go in a straight line if that is what is required.