Primary problem with my approach was that I didn't need to retrieve a token from ACS before I connect to the SB. Instead, I create a token by myself and and use that to connect to the SB. Basically, I created a SAML token and signed it with my certificate.
You will need to send the SAML token created above to ACS to gat back a SWT token.
This SWT token can be added to the message sent to Service Bus.