i am a strong believer that leaving the braces on for several months after the teeth are where you want them to be is a good thing. Stability comes into play.
Of course your retainer is part of your treatment and wearing it 24/7 is the key.... however a good ortho should be able to tell if your retainer came out good or not..if it is too tight it can cause issue.. or too loose...
Sometimes the person that make the molds for your retainers doesnt do the best job, so it is key to have an ortho that you can communicate all this with to keep your teeth in their position.
As you can see it is not uncommon to have braces on 2 or 3 times due to the shifting post braces.