When are user roles refreshed and how to force it?

First off, I'm not using FOSUserBundle and I can't because I'm porting a legacy system which has its own Model layer (no Doctrine/Mongo/whatsoever here) and other very custom behavior.

I'm trying to connect my legacy role system with Symfony's so I can use native symfony security in controllers and views.

My first attempt was to load and return all of the user's roles in the getRoles() method from the Symfony\Component\Security\Core\User\UserInterface. At first, it looked like that worked. But after taking a deeper look, I noticed that these roles are only refreshed when the user logs in. This means that if I grant or revoke roles from a user, he will have to log out and back in for the changes to take effect. However, if I revoke security roles from a user, I want that to be applied immediately, so that behavior isn't acceptable to me.

What I want Symfony to do is to reload a user's roles on every request to make sure they're up-to-date. I have implemented a custom user provider and its refreshUser(UserInterface $user) method is being called on every request but the roles somehow aren't being refreshed.

The code to load / refresh the user in my UserProvider looks something like this:

public function loadUserByUsername($username) {
    $user = UserModel::loadByUsername($username); // Loads a fresh user object including roles!
    if (!$user) {
        throw new UsernameNotFoundException("User not found");
    }
    return $user;
}

(refreshUser looks similar)

Is there a way to make Symfony refresh user roles on each request?