"I can totally understand the general idea of stock trading and how it helps both those who have good ideas and need money and those who have the money and invest it according to their taste."
This is a cognitive bias that is very prevalent (for obvious reasons) on HN. The markets are not primarily vehicles for moving money from investors to enterprises. Another important, and probably dominant, purpose of the markets is to accurately price and allow the buying and selling of risk. This is neither a new or unexpected phenomenon.
"But it's very hard for me to believe that something like HFT where firms go to unbelievable lengths to implement purely technological trading advantages benefit the real economy."
This is because of your cognitive bias and your ignorance (I mean this in the non-perjorative sense that you haven't investigated this). HFT firms, like any other trading participant, smooth demand curves generated either in time (buying from a participant now and selling later), venue (buying in one place and selling in another), or other ways. This allows other participants, who are not interested in being traders to hedge their risk more efficiently and thus more cheaply.
"The fact that financial industry as a whole just recently in the 2008/2009 crash and it's aftermath were able to extract obscene amounts of money from our governments (i.e. us) to keep the world's finance system from collapsing makes the suspection that most financial products purely based on a material advantage (be it monetary or technological) are akin to a scam very reasonable."
And now we see why gut feelings are dangerous in this discussion. You rightly feel deeply troubled by the big bailouts that cost all of us to the advantage of a very powerful few, and you correlate that with HFT. When in fact, HFT did not receive any bailouts and for the most part small independent shops without the size or power to engineer them. The 2008/2009 crash was based on non-HFT traded instruments where single deals could dwarf the entirety of the HFT industry.
What's particularly galling about this is that the anti-HFT narrative benefits the Goldmans and JPMs of the world. The largest technologically aggressive sell-side firms are smaller than the personal wealth of hedge fund principals.
Banks and hedge funds are both examples of buy-side firms. Confusing things further: the big investment banks all run operations that are morally equivalent to hedge funds.
This is a cognitive bias that is very prevalent (for obvious reasons) on HN. The markets are not primarily vehicles for moving money from investors to enterprises. Another important, and probably dominant, purpose of the markets is to accurately price and allow the buying and selling of risk. This is neither a new or unexpected phenomenon.
"But it's very hard for me to believe that something like HFT where firms go to unbelievable lengths to implement purely technological trading advantages benefit the real economy."
This is because of your cognitive bias and your ignorance (I mean this in the non-perjorative sense that you haven't investigated this). HFT firms, like any other trading participant, smooth demand curves generated either in time (buying from a participant now and selling later), venue (buying in one place and selling in another), or other ways. This allows other participants, who are not interested in being traders to hedge their risk more efficiently and thus more cheaply.
"The fact that financial industry as a whole just recently in the 2008/2009 crash and it's aftermath were able to extract obscene amounts of money from our governments (i.e. us) to keep the world's finance system from collapsing makes the suspection that most financial products purely based on a material advantage (be it monetary or technological) are akin to a scam very reasonable."
And now we see why gut feelings are dangerous in this discussion. You rightly feel deeply troubled by the big bailouts that cost all of us to the advantage of a very powerful few, and you correlate that with HFT. When in fact, HFT did not receive any bailouts and for the most part small independent shops without the size or power to engineer them. The 2008/2009 crash was based on non-HFT traded instruments where single deals could dwarf the entirety of the HFT industry.