IMHO, this is a broken promise about how the Web was supposed to be platform-independent. I get the feeling the general attitude on this thread is that such platforms are unimportant, but that doesn't mean it's not a broken promise.
I'm pretty sure many other fundamental protocols of the Web don't work so well on a non-2's-complement, non-8-bit-byte machine either.
Ignoring whether asm.js is a fundamental protocol or not, the Web wasn't supposed to depend desperately on the byte orientation of the consumer. Surely you remember how it was supposed to lift information access above what type of computer you viewed it on.
WebGL's endian specification was the beginning of the end, after which folks started saying certain implementational details weren't worth caring about, and this just finishes it off. And that's not what was being sold in the beginning.
I'm pretty sure many other fundamental protocols of the Web don't work so well on a non-2's-complement, non-8-bit-byte machine either.