Using LINQ to Objects to find items in one collection that do not match another

I want to find all items in one collection that do not match another collection. The collections are not of the same type, though; I want to write a lambda expression to specify equality.

A LINQPad example of what I'm trying to do:

void Main()
{
    var employees = new[]
    {
        new Employee { Id = 20, Name = "Bob" },
        new Employee { Id = 10, Name = "Bill" },
        new Employee { Id = 30, Name = "Frank" }
    };

    var managers = new[]
    {
        new Manager { EmployeeId = 20 },
        new Manager { EmployeeId = 30 }
    };

    var nonManagers =
    from employee in employees
    where !(managers.Any(x => x.EmployeeId == employee.Id))
    select employee;

    nonManagers.Dump();

    // Based on cdonner's answer:

    var nonManagers2 =
    from employee in employees
    join manager in managers
        on employee.Id equals manager.EmployeeId
    into tempManagers
    from manager in tempManagers.DefaultIfEmpty()
    where manager == null
    select employee;

    nonManagers2.Dump();

    // Based on Richard Hein's answer:

    var nonManagers3 =
    employees.Except(
        from employee in employees
        join manager in managers
            on employee.Id equals manager.EmployeeId
        select employee);

    nonManagers3.Dump();
}

public class Employee
{
    public int Id { get; set; }
    public string Name { get; set; }
}

public class Manager
{
    public int EmployeeId { get; set; }
}

The above works, and will return Employee Bill (#10). It does not seem elegant, though, and it may be inefficient with larger collections. In SQL I'd probably do a LEFT JOIN and find items where the second ID was NULL. What's the best practice for doing this in LINQ?

EDIT: Updated to prevent solutions that depend on the Id equaling the index.

EDIT: Added cdonner's solution - anybody have anything simpler?

EDIT: Added a var