You can create an SslStream. See here:
http://msdn.microsoft.com/en-us/library/system.net.security.sslstream%28VS.80%29.aspxAlthough the example is in C#, you hopefully can get the idea of how it works. Basically, you still use TcpListener to get the connection, then add an SslStream to the client connection using a cert and key-pair.