It really isn't the point that you can learn those things. People can also learn phone numbers and SSNs but we still use names. Wouldn't everything be much easier if we just referred to each other by our unique number? Addresses are also superfluous, just lat+lon with a unit # should be sufficient. Imperial measurements and their now American counterparts are based on human experience rather than something convenient with which to do calculations. Though the carpenters make a good case for 12in since it is so easily divided into 1/2, 1/3, 1/4, 1/6ths which are used often in construction. Even 1/5ths and 1/8ths go to only 1 decimal place. Metric is convenient for calculation and computing and not much more. Otherwise it is just a tradeoff you are making for a measurement system that wasn't designed around everyday life.
Take it from someone who grew up in a metric country - it makes absolutely no difference in everyday use. You are used to imperial units, so they make more sense to you. I'm not, so they feel weird, complicated and completely arbitrary. Typically I have to mentally convert to a metric measurement to have any intuition of how big or heavy something is.
In everyday use, I don't care if something is one and 3/5ths of some part of somebody's finger - I'm used to metric so it looks about 4 centimetres...
I'm not a builder, but I've never once heard any hint from my friends in construction that using millimetres has ever been a problem for them in their work, as they are used to it and the construction industry in every other country except the US is geared towards metric...