Absolutely not true. This is often referred to as the 'envelope' in construction and its not the majority of the cost of overall construction (or wasnt before the current outrageous lumber prices)