9
One worry (and it is a real worry) is cuckoo pointers.
Suppose you cache a pointer value A.
Then later free that pointer, then later allocate a new object B.
The memory allocator may choose the location that was used for A to allocate B.
Code of the form: if (new_pointer != old_pointer) {...}
may fail to realise that the object has changed.
http://www.nobugs.org/blog/archives/2004/11/20/cuckoo-pointers/
Edit:
Another way to put this, your object while alive will always be at the same address, the object at that address however will not always be the object you expect.
So long as the object stays in memory and you always refer to the same instance (eg. you don't pass it by value and then compare that object to the original or another cached copy) the address will always stay the same, and the pointer is a reasonable ID. This is obviously true -- if the object moved in memory, the pointer wouldn't work any more, and the standard guarantees that pointers to different objects compare to different values. [This is why you can't have zero length objects, so the following object in memory has a different address.]
However, if the object will ever be passed across a network or saved to disk, obviously that won't work any more, and you'll have to compare the object element-by-element, or have it include a unique id value (aka primary key).
ETA: borrible is correct that if you realloc the memory containing the objects, the values will be different, but obviously the pointers won't be valid at all, so your program will be broken whether you expect the pointers to be unique or not. (Conversely, if you use smart points that will track reacllocs -- essentially how C# and .NET work -- the value will not necessarily be the same, but the values of the pointers will still be different between different instances of the object.) In fact, that's the rule -- so long as the pointers are valid, you should be able to compare them, but you can't cache the value and expect it to still work some undetermined time later (in case you destroy the object first), so if you're sure you won't, it's ok, but keep it in mind, often you WILL want to serialise the objects at some point, and then having designed them like that from the start will be better.