none
My.Computer.Network.DownloadFile fails RRS feed

  • Question

  • The following line: My.Computer.Network.DownloadFile(strURL, fullname)

    fails with a strURL of:

    https://arxiv.org/pdf/1507.07998.pdf

    The message I get is:  The remote server returned an error: (403) Forbidden.

    Is this telling me that some protection has been put on by 'arxiv.org' so that I can't download the scientific paper there?

    Thanks.

    Sunday, June 17, 2018 10:36 AM

Answers

  • This works for me :

    Dim strURL As String = "https://arxiv.org/pdf/1507.07998.pdf"
    Using wc As WebClient = New WebClient()
        wc.Headers.Add("User-Agent: Test")
        wc.DownloadFile(strURL, "e:\\test.pdf")
    End Using

    • Edited by Castorix31 Sunday, June 17, 2018 10:57 AM
    • Marked as answer by Gidmaestro Sunday, June 17, 2018 9:45 PM
    Sunday, June 17, 2018 10:57 AM
  • Yes using web client as shown works for me.

    But these give the 403 error. I don't know why?

     'My.Computer.Network.DownloadFile("https://arxiv.org/pdf/1507.07998.pdf", "C:\test\t.pdf", "anonymous", "")
    
     My.Computer.Network.DownloadFile("https://arxiv.org/pdf/1507.07998.pdf", "C:\test\t.pdf")

    Because the server answer is :

    "Sadly, your client does not supply a proper User-Agent, and is consequently excluded.

    We have an inordinate number of problems with automated scripts which do not supply a User-Agent, and violate the automated access guidelines posted at 101 arxiv.org -- hence we now exclude them all.

    ...

    "

    • Marked as answer by Gidmaestro Sunday, June 17, 2018 9:45 PM
    Sunday, June 17, 2018 2:40 PM

All replies

  • This works for me :

    Dim strURL As String = "https://arxiv.org/pdf/1507.07998.pdf"
    Using wc As WebClient = New WebClient()
        wc.Headers.Add("User-Agent: Test")
        wc.DownloadFile(strURL, "e:\\test.pdf")
    End Using

    • Edited by Castorix31 Sunday, June 17, 2018 10:57 AM
    • Marked as answer by Gidmaestro Sunday, June 17, 2018 9:45 PM
    Sunday, June 17, 2018 10:57 AM
  • This works for me :

    Dim strURL As String = "https://arxiv.org/pdf/1507.07998.pdf"
    Using wc As WebClient = New WebClient()
        wc.Headers.Add("User-Agent: Test")
        wc.DownloadFile(strURL, "e:\\test.pdf")
    End Using

    Yes using web client as shown works for me.

    But these give the 403 error. I don't know why?

     'My.Computer.Network.DownloadFile("https://arxiv.org/pdf/1507.07998.pdf", "C:\test\t.pdf", "anonymous", "")
    
     My.Computer.Network.DownloadFile("https://arxiv.org/pdf/1507.07998.pdf", "C:\test\t.pdf")
    

    Sunday, June 17, 2018 11:20 AM
  • Off Topic

    Hmm. For some reason when I post lately I don't see my name as the last post n the main question listing page.

    I noticed yesterday after I posted again it appeared so lets see what  happens this is a test.


    Sunday, June 17, 2018 11:24 AM
  • Hmm. For some reason when I post lately I don't see my name as the last post n the main question listing page.

    I noticed yesterday after I posted again it appeared so lets see what  happens this is a test.

    Yes now my name appears as the last reply on the main page now.

    However when the was just my first answer my name did not appear.

    Same thing happened yesterday.

    PS Its possible this happens if you begin to make a post with the editor and then cancel. Then you start a new edit and make the post this time (without refreshing the page or leaving the page??).

    Sunday, June 17, 2018 11:28 AM
  • Yes using web client as shown works for me.

    But these give the 403 error. I don't know why?

     'My.Computer.Network.DownloadFile("https://arxiv.org/pdf/1507.07998.pdf", "C:\test\t.pdf", "anonymous", "")
    
     My.Computer.Network.DownloadFile("https://arxiv.org/pdf/1507.07998.pdf", "C:\test\t.pdf")

    Because the server answer is :

    "Sadly, your client does not supply a proper User-Agent, and is consequently excluded.

    We have an inordinate number of problems with automated scripts which do not supply a User-Agent, and violate the automated access guidelines posted at 101 arxiv.org -- hence we now exclude them all.

    ...

    "

    • Marked as answer by Gidmaestro Sunday, June 17, 2018 9:45 PM
    Sunday, June 17, 2018 2:40 PM
  • Off Topic. I guess maybe the forum just not showing correctly for everyone which it tends to do now and then. I see now Castor is not listed as thread last reply for this thread now.

    Just trying to help in case someone wants to look at it. :)

    PS Yes Reply no 5 seems missing from the listing.


    Sunday, June 17, 2018 5:01 PM