Actually, the Smalltalk community got it from Forth, where it was called "factoring" (no "re-"). See e.g. Leo Brodie's _Thinking Forth_ (http://thinking-forth.sourceforge.net/).
Well, Unit Tests also came from it (SUnit was the first Unit Test framework). Which is unsurprising, as refactoring without unit tests is quite daunting.
But JUnit did make them mainstream, and so did Eclipse' refactorings.
Interestingly, refactoring usually works quite well for functional Haskell code without tests just relying on the types. (Of course the types won't help you in more imperative sections of your code, i.e. it's hard to guarantee a specific order of actions in the IO Monad with types only.)