Haskell Linear Algebra Matrix Library for Arbitrary Element Types

I'm looking for a Haskell linear algebra library that has the following features:

• Matrix multiplication
• Matrix addition
• Matrix transposition
• Rank calculation
• Matrix inversion is a plus

and has the following properties:

• arbitrary element (scalar) types (in particular element types that are not Storable instances). My elements are an instance of Num, additionally the multiplicative inverse can be calculated. The elements mathematically form a finite field (_2²⁵⁶). That should be enough to implement the features mentioned above.
• arbitrary matrix sizes (I'll probably need something like 100x100, but the matrix sizes will depend on the user's input so it should not be limited by anything else but the memory or the computational power available)
• as fast as possible, but I'm aware that a library for arbitrary elements will probably not perform like a C/Fortran library that does the work (interfaced via FFI) because of the indirection of arbitrary (non Int, Double or similar) types. At least one pointer gets dereferenced when an element is touched
• (written in Haskell, this is not a real requirement for me, but since my elements are no Storable instances the library has to be written in Haskell)

I already tried very hard and evaluated everything that looked promising (most of the libraries on Hackage directly state that they wont work for me). In particular I wrote test code using:

• hmatrix, assumes Storable elements
• Vec, but the documentation states:

  Low Dimension : Although the dime