weak_ptr of a base class, while the shared_ptr is of a derived class?

I have a structure that manages objects that derive from a base class Entity, but does not control their lifetimes. I want this structure to be given weak pointers like weak_ptr<Entity> so that it can know if the object has been destroyed elsewhere.

However, outside of the managing structure where the shared pointer lives, I want the shared pointer to be the more specific shared_ptr<SpecificEntity> (SpecificEntity uses Entity as a base class).

Is there a way to accomplish this, or something like it?