In most cases, setting up your business there saves you no tax. It only saves you tax if you are a resident or if you don't tell the govt where you are a tax resident that you have a business there (not telling your govt is tremendously risky given that most of this information is now reported).
The main reason businesses are set up there is the legal benefits. Not just a stable legal environment but also because it is far easier to deal with international investors (most countries withhold tax from foreign investors at source, you can claim it back but that hits your ROI).
Depends on the country. Some countries have treaties letting your home corporation receive dividends from its foreign affiliate tax-free. Canada allows this kind of thing.
Then there's how to wash your Canadian income into this foreign operation...
There was a big case of a uranium mining company in Canada that sold options that effectively forced it to forever sell its product to its recently setup Swiss subsidiary at the "glut" price. If the price went up, all the profit would go to the Swiss subsidiary. If the price stayed low, they'd just liquidate the subsidiary.
The main reason businesses are set up there is the legal benefits. Not just a stable legal environment but also because it is far easier to deal with international investors (most countries withhold tax from foreign investors at source, you can claim it back but that hits your ROI).