YouTube - HttpWebRequests problem.


  • Please note that I am just practicing my HttpWebRequests knowledge. I am just trying to recreate the YouTube API. I will not use it or reproduce it.

    I have some weird problem with HttpWebRequests. I managed to login, but when I try to subscribe to some user I get this response from server: The remote server returned an error: (400) Bad Request.


    Imports System.Net
    Imports System.IO
    Imports System.Text
    Imports System.Text.RegularExpressions
    Imports System.Web
    Public Class Form1
     Dim F As New Functions
     Public Function GetBetween(ByRef strSource As String, ByRef strStart As String, ByRef strEnd As String, Optional ByRef startPos As Integer = 0) As String
     Dim iPos As Integer, iEnd As Integer, lenStart As Integer = strStart.Length
     Dim strResult As String
     strResult = String.Empty
     iPos = strSource.IndexOf(strStart, startPos)
     iEnd = strSource.IndexOf(strEnd, iPos + lenStart)
     If iPos <> -1 AndAlso iEnd <> -1 Then
      strResult = strSource.Substring(iPos + lenStart, iEnd - (iPos + lenStart))
     End If
     Return strResult
     End Function
     Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
       Dim Source As String = F.HttpGet("https://www.google.com/accounts/ServiceLogin?uilel=3&amp;service=youtube&amp;passive=true&amp;continue=http%3A%2F%2Fwww.youtube.com%2Fsignin%3Faction_handle_signin%3Dtrue%26nomobiletemp%3D1%26hl%3Den_US%26next%3D%252F&amp;hl=en_US&amp;ltmpl=sso")
    Dim DSH As String = vbNullString

    DSH = GetBetween(Source, "id=" & Chr(34) & "dsh" & Chr(34), " />")
    DSH = GetBetween(DSH, Chr(34), Chr(34))

    Dim GALX As String = vbNullString
    GALX = GetBetween(Source, "name=" & Chr(34) & "GALX" & Chr(34), " />")
    GALX = GetBetween(GALX, Chr(34), Chr(34))

    F.HttpPost("https://www.google.com/accounts/ServiceLoginAuth", "ltmpl=sso&pstMsg=&dnConn=&continue=http%3A%2F%2Fwww.youtube.com%2Fsignin%3Faction_handle_signin%3Dtrue%26nomobiletemp%3D1%26hl%3Den_US%26next%3D%252Findex&service=youtube&uilel=3&dsh=" + DSH + "&ltmpl=sso&hl=en_US&ltmpl=sso&timeStmp=&secTok=&GALX=" + GALX + "&Email=Username&Passwd=Password&rmShown=1&signIn=Sign+in&asts=", "application/x-www-form-urlencoded")

    Dim S As String = F.HttpGet("http://www.youtube.com/user/Chuubii")
    Dim Session_Token As String = GetBetween(S, "data-subscription-xsrf=" + Chr(34), "==") + "=="
    'Line under this is problem...
    F.HttpPost("http://www.youtube.com/subscription_ajax?action_create_subscription_to_user=1&feature=channel", "u=Chuubii&menu_type=button&session_token=" + Session_Token, "application/x-www-form-urlencoded")
    End Sub End Class

    Imports System.Net
    Imports System.IO
    Imports System.Text
    Public Class Functions
      Dim Cookies As New CookieContainer
      Public Function HttpGet(ByVal Address As String)
        Dim Request As HttpWebRequest = DirectCast(WebRequest.Create(Address), HttpWebRequest)
        Request.CookieContainer = Cookies
        Dim Response As HttpWebResponse = DirectCast(Request.GetResponse(), HttpWebResponse)
        Dim Reader As New StreamReader(Response.GetResponseStream())
        Dim Source As String = Reader.ReadToEnd
        Return Source
      End Function
      Public Function HttpPost(ByVal Address As String, ByVal PostData As String, ByVal ContentType As String)
        ServicePointManager.Expect100Continue = False
        Dim Encoding As New UTF8Encoding()
        Dim Bytes As Byte() = Encoding.GetBytes(PostData)
        Dim Request As HttpWebRequest = DirectCast(WebRequest.Create(Address), HttpWebRequest)
        Request.Method = "POST"
        Request.CookieContainer = Cookies
        Request.ContentType = ContentType
        Request.Referer = "http://www.google.com"
        Request.UserAgent = "Mozilla/5.0 (Windows; U; MSIE 7.0; Windows NT 6.0; en-US)"
        Request.ContentLength = Bytes.Length
        Dim RequestStream As Stream = Request.GetRequestStream()
        RequestStream.Write(Bytes, 0, Bytes.Length)
        Dim Response As HttpWebResponse
        Response = DirectCast(Request.GetResponse(), HttpWebResponse)
        Dim RequestReader As New StreamReader(Response.GetResponseStream)
        Dim Source As String = RequestReader.ReadToEnd
        Return Source
      End Function
    End Class


    Saturday, March 19, 2011 6:58 PM

All replies

  • BUMP. I really need this.
    Wednesday, March 23, 2011 7:36 PM
  • You are probably just missing something in your request.  You can use these posts to help you troubleshoot the issue:


    Use Fiddler http://fiddler2.com to compare the successful (IE) and unsucessful (your code) cases and make adjustments to your code from there.




    Jeff Sanders (MSFT)
    Thursday, March 24, 2011 5:32 PM
  • I've already tried to compare it with Fiddler, but I see nothing wrong. It would really mean a lot if someone could help me with this.
    Thursday, March 24, 2011 7:45 PM
  • It is not a question of something being wrong usually.  You will need to compare side by side the fiddler trace of you accessing the site with Internet Explorer, then your code.  HTTP has no magic to it and so if you send the same request (same headers, and body) you will get the same result.  If you see something different in the two traces, then you change your code to send the same exact thing that you see in the successful case.  What are the differences in the two traces?
    Jeff Sanders (MSFT)
    Thursday, March 24, 2011 7:55 PM
  • I found out what was the problem, but now it's even worse. Session token isn't displayed...


     Dim S As String = F.HttpGet("http://www.youtube.com/user/Chuubii")
    Dim Session_Token As String = GetBetween(S, "data-subscription-xsrf=" + Chr(34), "==") + "=="

    Basically, Session_Token is empty.


    Thursday, March 24, 2011 8:36 PM
  • So how does the session token get set in the IE case?  Where is this token in the HTML? (note you may have to clear all cookies in IE to see the session token set).
    Jeff Sanders (MSFT)
    Friday, March 25, 2011 12:05 PM
  • How did your troubleshooting go?  Were you able to compare the IE case with the managed code case?  I have done similar things a few times and every time the solution has been different.  Sometimes you miss an authentication piece that IE does for you automatically, and sometimes it is another issue like cookies.  If the authentication sends you a cookie you need to send that cookie with every request so you would need to create a global Cookie container to use with each request you issue (for example).  If you are under a tight deadline you can open a support case with Microsoft to assist you!
    Jeff Sanders (MSFT)
    Tuesday, March 29, 2011 1:51 PM
  • I didn't manage to solve this problem even after debugging with Fiddler. Where and how can I contact MS Support?
    Wednesday, April 06, 2011 8:36 PM
  • I am sorry you could not resolve this on your own.  You can contact support at this web site:


    Type Visual Studio in the Quick Product Finder box and select the version of Visual Studio that you are using.

    Then when you get to the Select Support Topic screen select Web Development, and the Sub-Topic will be WinHttp, WinINet, HttpWebRequest.

    Then complete the rest of the pages to submit a support case!


    Jeff Sanders (MSFT)
    Thursday, April 07, 2011 11:40 AM
  • What should I enter here?



    Thursday, April 07, 2011 5:44 PM
  • You should enter your access id

    If you don't have one, go back one screen and select this option: Don’t use a professional support license/contract (charges may apply)


    Jeff Sanders (MSFT)
    Thursday, April 07, 2011 5:48 PM
  • So, I have to pay like 100$ for solution?

    Where can I get the Access ID? MSDN, TechNet, or?

    Thursday, April 07, 2011 7:25 PM
  • If you have an MSDN or TechNet subscription you have an access id.  You should be able to find that in your account information.  If you do not have those subscriptions then you need to pay for support.
    Jeff Sanders (MSFT)
    Thursday, April 07, 2011 8:45 PM
  • Thank you. I didn't even know I can get a free MS support with TechNet/MSDN subscription.
    Thursday, April 07, 2011 9:02 PM
  • No problem!  If you have difficulty with your access code and opening a case, you can contact technet or MSDN for help.
    Jeff Sanders (MSFT)
    Friday, April 08, 2011 11:48 AM