Cannot get registration-free COM working from VBA

UPDATE: After spending 20+ hours trying to get a simple example working, I have realized that this is not as simple as it seems. Articles like this reveal the "gotchas" - and this was written before Windows 7 (which handles manifests differently). I'm sticking with exposing .NET assemblies to VBA via VSTO.

I made a simple COM-Visible .NET assembly and am trying to call it from Excel VBA. If I "Register for COM Interop" during my .NET build, I can call it from VBA successfully.

Sub VBA()    
    Dim obj As Object
    Set obj = actCtx.CreateObject("ComTest.Main")
    MsgBox obj.Test() '<-- Displays "Hello"
End Sub

However, I want to do registration-free.

Updated per advise from Hans:

I unchecked Register for COM Interop, and set my app.Manifest to:

<?xml version="1.0" encoding="utf-8"?>
<asmv1:assembly 
  manifestVersion="1.0" 
  xmlns="urn:schemas-microsoft-com:asm.v1" 
  xmlns:asmv1="urn:schemas-microsoft-com:asm.v1"
  xmlns:asmv2="urn:schemas-microsoft-com:asm.v2" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

  <assemblyIdentity
    type="win32"
    version="1.0.0.0" 
    name="ComTest" 
    publicKeyToken="a36a7110110d7bd7" />

  <clrClass
      clsid="{975DC7E0-4596-4C42-9D0C-0601F86E3A1B}"
      progid="ComTest.Main"
      threadingModel="Both"
      name="ComTest.Main"
      runtimeVersion="v4.0.30319">
  </clrClass>

  <file name = "ComTest.dll"></file>   
</asmv1:assembly>

I created a dummy "client.manifest" like this:

<?xml version="1.0" encoding="UTF-8" standalone=