locked
Reading Text files from another web RRS feed

  • Question

  • User1510859543 posted

    We have a public website that allows users to complete some information in a web form (name, email address, etc.) and store that data in a text file.  We want to read each of those Text files (*.txt) from a writeable virtual folder (UPLOADS) from our internal asp.net site.  Currently we are pulling images from that site using the code below. How do we code a way to loop through the Uploads folder and pull the txt file contents into our site?

            Dim client As HttpClient = New HttpClient()
            Dim strwebpath As String = "http://myweb.com/uploads/" & strFileName
    
            Dim bys As Task(Of Byte()) = client.GetByteArrayAsync(strwebpath)
            Using stream As MemoryStream = New MemoryStream(bys.Result)
                Using bmpImage As Bitmap = New Bitmap(stream)
                    bmpImage.Save(Server.MapPath(strSavePath))
                End Using
                stream.Dispose()
            End Using
            client.Dispose()

    Friday, July 19, 2019 9:06 PM

All replies

  • User475983607 posted

    Pretty simple.  Use the standard System.File.IO to get a listing of all the files.  Then loop over the files.

    https://docs.microsoft.com/en-us/dotnet/api/system.io.directory.getfiles?view=netframework-4.8

    Saturday, July 20, 2019 12:30 PM
  • User665608656 posted

    Hi dlchase,

    According to your requirements, I suggest that you can first generate the corresponding *.txt file in the UPLOADS folder on the corresponding web page, and then fetch all the *.txt files in the UPLOADS folder from another web page, and read the information in the UPLOADS folder separately, and display it.

    I did a small case based on this reuqirements. I created two identical WebForms page to fill in personal information and generate *.txt files.

    On the third page, I read and output all the *.txt files by the button to the current page.

    For more details, you could refer to the following code:

    WebForm_info1.aspx (WebForm_info2.aspx) :

    <!DOCTYPE html>
    
    <html xmlns="http://www.w3.org/1999/xhtml">
    <head runat="server">
        <title></title>
    </head>
    <body>
        <form id="form1" runat="server">
            this is info1 page
            <div>
                <table style="width: 200px;">
                    <tr>
                        <td>Name</td>
                        <td>
                            <asp:TextBox ID="TextBox1" runat="server"></asp:TextBox></td>
                    </tr>
                    <tr>
                        <td>Email</td>
                        <td>
                            <asp:TextBox ID="TextBox2" runat="server"></asp:TextBox></td>
                    </tr>
                    <tr>
                        <td>Address</td>
                        <td>
                            <asp:TextBox ID="TextBox3" runat="server"></asp:TextBox></td>
                    </tr>
                </table>
                <asp:Button ID="Button1" runat="server" Text="save" OnClick="Button1_Click" /> 
            </div>
        </form>
    </body>
    </html>

    WebForm_info1.aspx.cs(WebForm_info2.aspx.cs) :

    Protected Sub Button1_Click(ByVal sender As Object, ByVal e As EventArgs)
        Try
            Dim path As String = "C:\Users\yongqy\source\repos\CaseProject\WebApplication_July\Uploads\Info1.txt"
    
            If Not File.Exists(path) Then 
                Using tw As StreamWriter = File.CreateText(path)
                    tw.WriteLine("Name:" & TextBox1.Text.ToString())
                    tw.WriteLine("Email:" & TextBox2.Text.ToString())
                    tw.WriteLine("Addresss:" & TextBox3.Text.ToString())
                End Using
            End If
    
            Dim builder As StringBuilder = New StringBuilder()
            builder.Append("<script language=JavaScript>")
            builder.Append("alert(""Create Info1 successfully!"")")
            builder.Append("</script>")
            Page.ClientScript.RegisterStartupScript(Me.[GetType](), "SuccessAlert", builder.ToString())
        Catch ex As Exception
            Throw ex
        End Try
    End Sub

    WebForm_readfiles.aspx.cs :

    Protected Sub Button1_Click(ByVal sender As Object, ByVal e As EventArgs)
        Dim path As String = "C:\Users\yongqy\source\repos\CaseProject\WebApplication_July\Uploads\"
    
        For Each file As String In Directory.EnumerateFiles(path, "*.txt")
            Dim lines As String() = File.ReadAllLines(file)
    
            For Each line As String In lines
                Label1.Text += line & "<br/>"
            Next
    
            Label1.Text += "<br/>"
        Next
    End Sub

    The result of this work demo:

    Best Regards,

    YongQing.

    Monday, July 22, 2019 5:47 AM
  • User1510859543 posted

    Where does the reading of text files from the "external" web come from? e.g. http://myweb/mysite/uploads

    Tuesday, July 23, 2019 12:27 PM
  • User475983607 posted

    Where does the reading of text files from the "external" web come from? e.g. http://myweb/mysite/uploads

    The external web site is responsible for exposing the files.

    Tuesday, July 23, 2019 1:09 PM
  • User665608656 posted

    Hi dlchase,

    Where does the reading of text files from the "external" web come from?

    The location of the text files retrieved from an external website is where you store the text file on another page.

    And the location of text file storage is up to you.

    Best Regards,

    YongQing.

    Wednesday, July 24, 2019 9:25 AM
  • User753101303 posted

    Hi,

    Seems the real issue is to list which text files are available ? It is not stored in a folder you could access through a network share ? Else you'll need a way to make this list available (at worst WebDAV but it is not likely not the first option).

    Also make sure about security. It's unclear if those text files are then freely available on the web knowing their  name or really protected.

    Wednesday, July 24, 2019 9:47 AM
  • User1510859543 posted

    Below is what I tried but am getting error "URI formats are not supported" on start of directory loop of files. Is there another way of getting a list of the files?

            Dim client As HttpClient = New HttpClient()
            Dim strwebpath As String = "http://mytestweb.com/uploads/"
    
            For Each s In Directory.GetFiles(strwebpath, "*.txt")
                Dim fi As FileInfo = New FileInfo(s)
                If Right(s, 9) <> "Thumb.jpg" Then
                    If fi.Extension <> "" Then
                        ListBox1.Attributes.Add(fi.Name, fi.Name)
                    End If
                End If
            Next

    Wednesday, July 24, 2019 1:27 PM
  • User475983607 posted

    Below is what I tried but am getting error "URI formats are not supported" on start of directory loop of files. Is there another way of getting a list of the files?

            Dim client As HttpClient = New HttpClient()
            Dim strwebpath As String = "http://mytestweb.com/uploads/"
    
            For Each s In Directory.GetFiles(strwebpath, "*.txt")
                Dim fi As FileInfo = New FileInfo(s)
                If Right(s, 9) <> "Thumb.jpg" Then
                    If fi.Extension <> "" Then
                        ListBox1.Attributes.Add(fi.Name, fi.Name)
                    End If
                End If
            Next

    The remote server must expose the files or code to get a list of files.  For example, if the remote server allow directory browsing or exposes the file from a URL.

    Wednesday, July 24, 2019 1:44 PM
  • User1510859543 posted

    Since we also control the remote site, is there a way that when we create(save) the file locally on the remote server to instead save that file to a remote site?

    Wednesday, July 24, 2019 1:51 PM
  • User753101303 posted

    Directory.GetFiles is for file system access. You can't list files this way through http.

    This is why I was trying first to,understand if your server could access to this location using a network share or if using http is really your only option. In this case, you'll still have to keep the list of files somewhere (a db or maybe another text file at the same location or maybe a web service)

    Wednesday, July 24, 2019 1:55 PM
  • User475983607 posted

    Since we also control the remote site, is there a way that when we create(save) the file locally on the remote server to instead save that file to a remote site?

    I honestly have no idea what you are asking.  Save file locally on the remote server?  What does that mean?

    Perhaps explain the programming problem rather than how you think the problem should be solved.

    Wednesday, July 24, 2019 1:58 PM
  • User1510859543 posted

    We have a website at a remote location that allows people to upload photos to a virtual directory on that remote server.  The process also creates a text file (.txt) with photo information in it.  We want to be able to read the text files from our local web server and process that text and pull the photos into our local server.

    We have been using email attachments and want to get away from that.

    Wednesday, July 24, 2019 2:08 PM
  • User475983607 posted

    dlchase

    We have a website at a remote location that allows people to upload photos to a virtual directory on that remote server.  The process also creates a text file (.txt) with photo information in it.  We want to be able to read the text files from our local web server and process that text and pull the photos into our local server.

    We have been using email attachments and want to get away from that.

    Create a web page, on the remote server with the files, that returns the data formatted that is easy for you to handle.   For example, return a comma separated list, XML, or JSON.  Loop over the data and fetch the files. 

    If the two systems are on the same network then use a network shard.  There's also FTP or SFTP and a scheduled task.

    Wednesday, July 24, 2019 2:26 PM
  • User1510859543 posted

    Create a web page, on the remote server with the files, that returns the data formatted that is easy for you to handle.   For example, return a comma separated list, XML, or JSON.  Loop over the data and fetch the files. 

    How would this be done from the remote application that files are uploaded to?

    Wednesday, July 24, 2019 3:07 PM
  • User475983607 posted

    How would this be done from the remote application that files are uploaded to?

    You have to write code.  Are you asking how to make a list of file names?

    Wednesday, July 24, 2019 3:20 PM
  • User753101303 posted

    You really have no other way to talk between your servers than http (they are not in the same network ?).

    If confirmed you could maybe write those file names to a known location and so you'll read first this file to know which other files could be downloaded (quick but dirty).

    Depending on your needs and if you need to further extend this you could have a look at https://docs.microsoft.com/en-us/aspnet/web-api/overview/getting-started-with-aspnet-web-api/tutorial-your-first-web-api and HttpClient (web services) which allows to expose data/capabilities over http.

    Thursday, July 25, 2019 8:58 AM
  • User1510859543 posted

    So, could I just write the text file to a virtual directory on a remote website using HttpClient?  If so, how is that done?  I have used below to read a file remotely.

    Private Sub SurroundingSub()
        Dim fileLocation As String = "http://site/file.txt"
        Dim client As WebClient = New WebClient()
        Dim stream As Stream = client.OpenRead(fileLocation)
        Dim reader As StreamReader = New StreamReader(stream)
        Dim content As String = reader.ReadToEnd()
    End Sub

    Friday, July 26, 2019 11:35 AM
  • User665608656 posted

    Hi dlchase,

    According to your description, what's the relationship between the remote server and local server? 

    If the remote server and local server is in same domain we could use share folder to access the file.

    You could refer to this link: Share a Folder or Drive

    Best Regards,

    YongQing.

    Monday, July 29, 2019 7:56 AM
  • User1510859543 posted

    The local server is not on the same domain as the remote server.

    Monday, July 29, 2019 1:56 PM
  • User665608656 posted

    Hi dlchase,

    In this way, you need to put your remote server side file into IIS to create an application or use FTP to transfer files between different domains.

    You can refer to this link how to use ftp : https://kb.iu.edu/d/aerg

    Here is a link about how to read file in ftp : Read file from FTP to memory in C#

    Best Regards,

    YongQing.

    Tuesday, July 30, 2019 10:13 AM
  • User1510859543 posted

    Yes, that works great when you know the name of the files but we don't know them.  We just want to get all of the *.txt files and process them one at a time.  I think the ftp will work for pulling the images though.

    Tuesday, July 30, 2019 12:46 PM
  • User475983607 posted

    dlchase

    Yes, that works great when you know the name of the files but we don't know them.  We just want to get all of the *.txt files and process them one at a time.  I think the ftp will work for pulling the images though.

    This has been explained several times.  Write code that returns a list of files.   Deploy the code to the remote web application.  Invoke the the code using a simple HTTP GET.   This is a very simple programming task for someone with your forum experience.   The task should take a few minutes to code.

    Add a generic handler to the remote site using the standard Visual Studio templates.

    using Newtonsoft.Json;
    using System.IO;
    using System.Web;
    
    namespace WebFormsDemo
    {
        /// <summary>
        /// Summary description for GetFiles
        /// </summary>
        public class GetFiles : IHttpHandler
        {
            public void ProcessRequest(HttpContext context)
            {
                //Get files within a folder
                string targetDirectory = context.Server.MapPath("~/Scripts");
                string[] fileEntries = Directory.GetFiles(targetDirectory);
    
                //NewtonSoft to serialize the folder list
                string json = JsonConvert.SerializeObject(fileEntries);
    
                //Return the json result to the caller.
                context.Response.ContentType = "application/json";
                context.Response.Write(json);
            }
    
            public bool IsReusable
            {
                get
                {
                    return false;
                }
            }
        }
    }

    Deploy the ashx file.  Then test the handler using your browser.

    http://localhost:51867/getfiles.ashx

    Next write code on the client to fetch the file list.  I'll leave that up to you...

    Tuesday, July 30, 2019 12:57 PM