This is because the client library (SqlClient, ODBC, JDBC, for example) keeps a connection pool underneath. Basically, the first connection takes more time because goes through the whole aspects of the connection. Once released by the application,
it is kept alive in the background and next time there is a connection attempt, this connection is re-used.
Please, let me know if you have more questions.
Thanks
Luiz Santos