I'm trying to do the same setup. We are running Websphere and Domino with the SSO key generated by websphere. I've tried the 3 keys offered by the LTPA export from websphere, with no success.
I think that there might be an other problem. The LTPA key is given to the browser when the user login to websphere. The key can be used to pass to domino sites. This doesn't work the other way around. If I login to domino, I still have to login to websphere.
Can this be solved ?