When I lived in the UK for a bit, iirc people would use the term "builder" where I would say "contractor".
At least in American English vernacular, saying "he's a contractor" either means he builds houses, or he's an independent contractor (doing some "consulting-ish" work). The lack of context for that phrase is indeed confusing!
Most of the UK population, if asked "what's a contractor" would assume construction.