creating macros in intellij

I am trying to create a macro that takes a selection of text and runs some find/replace on it using a regex. I have recorded this macro in intellij, but when I run it, all I get is the find/replace dialog opened. No find/replace is executed.

Has anyone had any luck with creating a find/replace macro in intellij?