Am I right in saying that const_cast followed by modification on a ref-to-const bound to a temporary is okay?

I would like to check my understanding and conclusions on this matter.

On IRC, it was asked:

Is it acceptable to const_cast a const reference that's bound to a temporary object?

Translating: he has a ref-to-const bound to a temporary, and he wants to cast away its const-ness to modify it.

My response was that I'd asked a similar question previously, where the consensus seemed to be that temporaries themselves are not inherently const, and thus that you can cast off the const-ness of a reference you have to them, and modify them through the result. And, as long as that original ref-to-const still exists, this won't affect the temporary's lifetime.

That is:

int main()
{
   const int& x = int(3);

   int& y = const_cast<int&>(x);
   y = 4;

   cout << x;
}
// Output: 4
// ^ Legal and safe

Am I right?

^{(Of course, whether or not such code is actually advisable is another matter entirely!)}