First up, a simplified version of the task I want to accomplish: I have several large files (amounting to 30GB) that I want to prune for duplicate entries. To this end, I establish a database of hashes of the data, and open the files one-by-one, hashing each item, and recording it in the database and the output file iff its hash wasn't already in the database.
I know how to do this with iteratees, enumerators, and I wanted to try conduits. I also know how to do it with conduits, but now I want to use conduits & persistent. I'm having problems with the types, and possibly with the entire concept of ResourceT.
Here's some pseudo code to illustrate the problem:
withSqlConn "foo.db" $ runSqlConn $ runResourceT $
sourceFile "in" $= parseBytes $= dbAction $= serialize $$ sinkFile "out"
The problem lies in the dbAction function. I would like to access the database here, naturally. Since the action it does is basically just a filter, I first thought to write it like that:
dbAction = CL.mapMaybeM p
where p :: (MonadIO m, MonadBaseControl IO (SqlPersist m)) => DataType -> m (Maybe DataType)
p = lift $ putStrLn "foo" -- fine
insert $ undefined -- type error!
return undefined
The specific error I get is:
Could not deduce (m ~ b0 m0)
from the context (MonadIO m, MonadBaseControl IO (SqlPersist m))
bound by the type signature for
p :: (MonadIO m, MonadBaseControl IO (SqlPersist m)) =>
DataType -> m (Maybe DataType)
at tools/clean-wac.hs:(33,1)-(34,34)
`m' is a rigid type variable bound by
the type signature for
p :: (MonadIO m, MonadBaseControl IO (SqlPersist m)) =>
DataType -> m (Maybe (DataType))
at tools/clean-wac.hs:33:1
Expected type: m (Key b0 val0)
Actual type: b0 m0 (Key b0 val0)