Java Security

I'm creating a trust manager that logs (among other things) info about a connection if the server fails the default trust manager isServerTrusted() call. At the time I register my trust manager with the ssl context, it's when then client socket factory is created so I have no reference to a socket. Does anybody know a way to retrieve the active socket info from within the trust manager when the socket is created? I can certainly register a callback handler in my trust manager, but I still have the issue of matching it up with any active sockets so that I can grab the ip address, etc.

Regards,
Bill