You've described a condition called "Alcohol Flush Reaction". This condition is due to an inactive enzyme which is normally responsible for the breakdown of acetaldehyde, a by product of the metabolism of alcohol. Acetaldehyde is a toxin from alcohol. Because the body is unable to break it down, the accumulation of the poison in the blood stream causes flushing (redness in face and chest). This condition (inactive enzyme) is said to be more prevalent amongst the Asian community and is attributed to their lower levels of alcoholism; due to the fact that the reaction prompts them to stop the abuse of alcohol by nonconsumption of alcohol.
Alcohol flush reaction (Asian Flush, Asian Glow, Red Cheeks, Oriental Flush, Big Blush) is a condition in which the body cannot break down ingested alcohol completely, due to a missense polymorphism that encodes the enzyme, acetaldehyde dehydrogenase (ALDH2) , normally responsible for breaking down acetaldehyde, a product of the metabolism of alcohol. Flushing, or blushing, is associated with the erythema (reddening caused by dilation of capillaries) of the face, neck, shoulder, and in some cases, the entire body after consumption of alcohol.
Okay, this is why I don't usually drink...and the answer is not in 'building up' tolerance, it's not going to happen!