Using custom User admin breaks change password form in Django's admin

I'm using a custom User admin by:

class CustomUserAdmin(admin.ModelAdmin):
    model = User
    ...
admin.site.unregister(User)
admin.site.register(User, CustomUserAdmin)

but when I try to change the password via the admin page I get a 404.

user object with primary key u'4/password' does not exist.

Reverting back to the default User admin works fine.