Using a different port should be possible. You have to specify the "servername:port" in the edit box in SSAdmin, server/configure/internet page.
The same port number will be used for both http/https, so pay attention to your SSL settings on the server (in SSAdmin, Server/Configure/Internet) and the client (in VS, Tools/Options/SourceControl/PluginSettings/Advanced). The "An Error Occured in the Secure Channel Support" means that VS did either a http request on the port used by SSL, or a https request on the port used by non-SSL web.
Alin