How to cast a value of generic type T to double without boxing?

Imagine the following simple code:

public void F<T>(IList<T> values) where T : struct
{
  foreach (T value in values)
  {
    double result;
    if (TryConvertToDouble((object)value, out result))
    {
      ConsumeValue(result);
    }
  }
}

public void ConsumeValue(double value)
{
}

The problem with the above code is casting to object, which results in boxing in the loop.

Is there a way to achieve the same functionality, i.e. feeding ConsumeValue with all the values without resorting to boxing in the foreach loop? Note, that F must be a generic method.

I can live with an expensive preparation code as long as it is executed outside the loop just once. For instance, if a fancy dynamic method needs to be emitted, then it is fine if done just once.

Edit

T is guaranteed to be of some numeric type or bool.

Motivation. Imagine meta data driven application, where an agent reports a data stream, where data item type is dynamically emitted based on the data stream meta data. Imagine also, that there is normalizer engine, which knows to normalize numeric data streams according to some algorithm. The type of the incoming numeric data stream is known only at run time and can be directed to a generic method of that data type. The normalizer, however, expects doubles and produces doubles. This is just a very high level description.

Edit 2

Concerning the cast to double. Actually we have a method to convert to double with the following signature:

bool TryConvertToDouble(object value, out double result);

I should have used it in the example in the first place, but I wanted to save space and written something that is not going to work. Fixed it now.

Edit 3

The current implementation does box the values. And even if I do not have the profiler's verdict as to performance penalty of it (if any), still I am interesting to know whether there