Well, nothing demonstrates that they're the same people. Strictly, there could be a chunk of the populace that shifts their allegiance to whoever is more spy-happy.

... don't think that's actually the case, though.