how to respond to an ajax call in before_filter

I write a before_filter like this:

def authenticate_user
  if request.xhr?
    flash.now[:alert] = 'Error'
    render :partial => "js_helpers/popover", :formats => :js
  else
    authenticate_user!
  end
end

However, it doesn't work as expected. The log file shows:

  Rendered js_helpers/_popover.js.erb (0.1ms)
Filter chain halted as :authenticate_user rendered or redirected

The function of the js code is to show a popover block showing flash message. but it looks like the js code is not excited at all.

So, I changed my way:

def authenticate_user
  if request.xhr?
    flash.now[:alert] = 'Error'
    render :js => "window.location = '/users/sign_up'"
  else
    authenticate_user!
  end
end

Instead, I want to redirect to sign up page.

However, there is still problems. the flash seems not work

How can I achieve my goal:

when the request is an ajax call, execute come javascript code or redirect_to a new page. Under both condition, I need to store some message in flash.

Plus, the javascript code is very long so that I need to put it in a file. How can I use render :js to execute a javascript inside a file?