How can I convert a DataTable to an IEnumerable<Dictionary<string, object>>?

I'd like to convert a DataTable to an IEnumerable<> of Dictionary<string, object>. I tried the following LINQ query,

from DataRow row in ds.Tables[0].AsEnumerable()
let rowDictionary = new Dictionary<string, object>()
from DataColumn column in row.Table.Columns.Cast<DataColumn>()
select rowDictionary.Add(column.ColumnName, row[column]).ToArray();

but I get the following error:

error CS1943: An expression of type 
'System.Collections.Generic.IEnumerable<System.Data.DataColumn>' is not 
allowed in a subsequent from clause in a query expression with source type 
'System.Data.EnumerableRowCollection<AnonymousType#1>'.  Type inference 
failed in the call to 'SelectMany'.

I know I can brute-force this with a loop, but it seems like something I should be able to do in LINQ. Thanks in advance for any help!