Copying from one table to another to create a reverse index


  • Good Day,
    I am trying to figure out the best way to create a reverse lookup index of my data. Basically I have data in Table A which I need to reverse into Table B so that Table B has a partition key matching Table A's RowKey and Table B's RowKey matches Table A's Partition Key. Something like this:
    Table A:
    PK / RK
    A / 1
    A / 2
    A / 3
    B / 1
    B / 2
    PK / RK
    1 / A
    1 / B
    2 / A
    2 / B
    3 / A
    I am using C# and the Microsoft.WindowsAzure.Storage.Table libraries at this time, but open to all options. Most important is that it needs to happen quickly and be executable by code as needed. Since these tables are created and dropped regularly as part of the job process.
    Copying from one table to another
    marți, 12 iunie 2018 15:57

Toate mesajele

  • Just for clarification are you looking for index custom columns in azure table storage?

    Refer :

    marți, 12 iunie 2018 20:54
  • Sandeep,

    I am looking for a quick / efficient way to copy data from one table to another table while assigning columns from table a to the partition and row key of table b.  Thus I will have 2 copies of the data (which is fine because it's very small row size) but each copy will use a different partition and rowkey.  Thus allowing me to scan millions of records much more quickly.

    What I don't want to do, is have to retrieve records from table a 1000 at a time, try and store them all in memory and then push them all back up once I have them all down and can properly batch them for return.

    If this was SQL then it would look something like this:

    INSERT INTO TableB (PartitionKey, RowKey, Data1, Data2, Data3) SELECT Rowkey as 'PartitionKey', PartitionKey as 'RowKey', Data1, Data2, Data3 FROM TableA

    Does that make sense?

    • Editat de ataylorm1 marți, 12 iunie 2018 23:14
    marți, 12 iunie 2018 23:10
  • Bump!
    vineri, 15 iunie 2018 17:53
  • you could consider using the thread discussed in SO:
    you'd have to recycle the code by specifying the row keys(not sure if this is programmatically feasible):

    private IEnumerable<T> FetchAllEntities()
        List<T> allEntities = new List<T>();
        CloudStorageAccount storageAccount = CloudStorageAccount.DevelopmentStorageAccount;
        CloudTable table = storageAccount.CreateCloudTableClient().GetTableReference("MyTable");
        Microsoft.WindowsAzure.Storage.Table.TableContinuationToken tableContinuationToken = null;
            var queryResponse = table.ExecuteQuerySegmented<T>(null, tableContinuationToken, null, null);
            tableContinuationToken = queryResponse.ContinuationToken;
        while (tableContinuationToken != null);
        return allEntities;
    Let us know if this helps.

    marți, 19 iunie 2018 23:01
  • Hey ,

    Just checking in to see if the above suggestions helped or you need further assistance on this issue.

    joi, 21 iunie 2018 18:54