locked
Multiple Polybase tables - One data source RRS feed

  • Question

  • I am trying to achieve 65 unique external tables using a single Azure BLOB container full of CSV files.

    There are 65 files in the root i.e (file1,file2,file3..,file65), there are sub-folders (about 200 subfolders) with the same named files (/folder/file1,/folder/file2,/folder/file3..,/folder/file65) inside.

    Each file named file1 is identical in structure as the other files named file1 and so on. My question is, is there a way I can create 65 Tables with Polybase where each table only references to each particular file name?

    I'm interested to see what thoughts there are, pre-processing / re-arranging the file structure isn't really an option either.

    Thanks 

    Tuesday, November 19, 2019 8:31 PM

All replies

  • Hi BradleyHampson,

     

    Did you mean that an external table corresponds to an external file? And all these external files belong to a data source>

     

    Best regards,

    Dedmon Dai


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com

    Wednesday, November 20, 2019 9:43 AM
  • Hi Dedmon,

    I would like the tables to look like this. All files are part of the same datasource (BLOB Storage)

    Table1 -> External(File1, folder1-File1, folder2-File1, folder3-File1, folder4-File1)

    Table2 -> External(file2, folder1-file2, folder2-file2, folder3-file2, folder4-file2)

    Table3 -> External(file3, folder1-file3, folder2-file3, folder3-file3, folder4-file3)

    Hope this makes sense. Thanks for the response.

    Wednesday, November 20, 2019 10:17 AM
  • Hi BradleyHampson,

     

    Sorry for late response, In general, an external table corresponds to an external file, and the example of the official document is the same. In this case, I need to test it.

     

    Best regards,

    Dedmon Dai


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com

    Monday, November 25, 2019 9:35 AM