Creating a dictionary from another dictionary using LINQ

I have a dictionary of the type:

IDictionary<foo, IEnumerable<bar>> my_dictionary

bar class looks like this:

class bar
{
    public bool IsValid {get; set;} 
}

How can I create another dictionary with only those items that have IsValid = true.

I tried this:

my_dictionary.ToDictionary( p=> p.Key,
                            p=> p.Value.Where (x => x.IsValid));

The problem with above code is that this creates a key with empty enumerable, if all the elements for that key were IsValid = false.

for example:

my_dictionar[foo1] = new List<bar> { new bar {IsValid = false}, new bar {IsValid = false}, new bar {IsValid = false}};
my_dictionary[foo2] = new List<bar> {new bar {IsValid = true} , new bar{IsValid = false};
var new_dict = my_dictionary.ToDictionary( p=> p.Key,
                            p=> p.Value.Where (x => x.IsValid));
// Expected new_dict should contain only foo2 with a list of 1 bar item.
// actual is a new_dict with foo1 with 0 items, and foo2 with 1 item.

How do I get my expected.