Replaces old configuration provider with new.
The given old configuration provider must have been configured already.