none
How to Copy from an existing blob into a new Asset

    Question

  • A number of folks on the forum have contacted me and asked how to copy from an existing blob in a storage account into a WAMS Asset.

    Nick Drouin wrote some code that was posted to a recent thread that I wanted to share for folks looking for examples on how to do this.  I moved it here so that you could find it easily when searching the Forum.

    The code below uses the CopyBlob operation in the StorageClient library to quickly copy a blob from an existing file into a newly created "empty" Asset in Media Services so that you can use it in any encode job workflow or Origin streaming (if it is Smooth Streaming format already).

     

    Limitations on the code sample.

    • the sample code below only copies a single file into the asset.
    • Only tested this while copying to/from the same storage account, the one registered with WAMS.

     

    <PREVIEW CODE REMOVED - SEE BELOW>

    Sunday, July 29, 2012 10:56 PM
    Owner

Answers

All replies

  • Also check out this MSDN Blog that talks about the introduction of the new Cross-Account Copy Blob operations in version 1.7.1 of the Storage Client SDK.

    I tested this out successfully also by downloading the latest source code for the storage client from GITHub, compiling it and adding it to my project.

    Once you have version 1.7.1 of the Storage Client, you will find the new StartCopyFromBlob method on the CloudBlob class.   This will allow you to copy across storage accounts.  I successfully used this to import a video file into a new empty Asset in WAMS from my US-EAST storage account, directly into my US-WEST storage account that is configured for Media Services.  You could also use this API call in reverse if you wanted to copy the contents of an Asset in Media Services to another Storage Account location.


    Monday, July 30, 2012 1:28 AM
    Owner
  • Here is some updated code for the above.

    It uses the NuGet installpackage WindowsAzure.MediaServices -Version 2.0.0.5

            public void UseAzureStorageSdkToUpload()
            {
                //
                //Create an empty asset:
                Guid g = Guid.NewGuid();
                IAsset assetToBeProcessed = _context.Assets.Create("YourAsset_" + g.ToString(), AssetCreationOptions.None);
    
                //
                //Create a locator to get the SAS url:
                IAccessPolicy writePolicy = _context.AccessPolicies.Create("Policy For Copying", TimeSpan.FromMinutes(30), AccessPermissions.Write | AccessPermissions.List);
                ILocator destinationLocator = _context.Locators.CreateSasLocator(assetToBeProcessed, writePolicy, DateTime.UtcNow.AddMinutes(-5));
    
                //
                //Create CloudBlobClient:
                var storageAccountName = ConfigurationManager.AppSettings["storageAccountName"];
                var storageAccountKey = ConfigurationManager.AppSettings["storageAccountKey"];
                var storageInfo = new StorageCredentialsAccountAndKey(storageAccountName, storageAccountKey);
                CloudBlobClient cloudClient = new CloudBlobClient("http://" + storageAccountName + ".blob.core.windows.net", storageInfo);
    
                //
                //Create the reference to the destination container:
                string destinationContainerName = (new Uri(destinationLocator.Path)).Segments[1];
                CloudBlobContainer destinationContainer = cloudClient.GetContainerReference(destinationContainerName);
    
                //
                // We will set these differently based on how we are getting the files into the asset:
                var destinationFileBlobName = "";
                long destinationFileLength = 0;
    
    
                //Depending on your use-case, you may be moving something your already have, or uploading from disk.
                //Use the following switch to try out the two:
                bool copyFromExistingBlob = true;
    
    
                if (copyFromExistingBlob)
                {
                    //
                    //Specific things you'll need to set:
                    var sourceContainerName = "uploads";
                    var sourceFileBlobName = "FileToCopy.mp4";
    
                    destinationFileBlobName = sourceFileBlobName;
                    
                    //
                    //Create the reference to the source container, in this case, a container called uploads:
                    CloudBlobContainer sourceContainer = cloudClient.GetContainerReference(sourceContainerName);
    
                    //
                    //Get and validate the source blob, in this case a file called FileToCopy.mp4:
                    CloudBlob sourceFileBlob = sourceContainer.GetBlobReference(sourceFileBlobName);
                    sourceFileBlob.FetchAttributes();
                    long sourceFileLength = sourceFileBlob.Properties.Length;
                    if(sourceFileLength == 0) 
                        throw new FileNotFoundException(string.Format("The source {0} file does not exist or has a file size of zero.", sourceFileBlobName));
    
                    //If we got here then we can assume the source is valid and accessible.
                    
                    //
                    //Create destination blob for copy, in this case, we choose to rename the file:
                    CloudBlob destinationFileBlob = destinationContainer.GetBlobReference(destinationFileBlobName);
    
                    //
                    //Do the blob-to-blob copy:
                    destinationFileBlob.CopyFromBlob(sourceFileBlob);
                        // Will fail here if project references are bad (the are lazy loaded).
    
                    //
                    //Check destination blob:
                    destinationFileBlob.FetchAttributes();
                    destinationFileLength = destinationFileBlob.Properties.Length;
                    if(destinationFileLength != sourceFileLength)
                        throw new Exception(string.Format("The copied file {0} does not have the same size as the source file {1}.", sourceFileBlobName, destinationFileBlobName));
    
                    //If we got here then the copy worked.
                }
                else //upload it from disk via the Azure Storage SDK functions
                {
                    //
                    //Specific things you'll need to set:
                    var sourceFilePath = @"c:\temp\uploads\1.mp4";
    
    
                    var sourceFileLength = (new FileInfo(sourceFilePath)).Length; //this will throw for invalid file paths.
                    destinationFileBlobName = Path.GetFileName(sourceFilePath);
        
                    //
                    //Create destination blob for copy, in this case, we choose to rename the file:
                    CloudBlob destinationFileBlob = destinationContainer.GetBlobReference(destinationFileBlobName);
    
                    //
                    //Do the upload:
                    destinationFileBlob.UploadFile(sourceFilePath);
                    
                    //
                    //Check destination blob:
                    destinationFileBlob.FetchAttributes();
                    destinationFileLength = destinationFileBlob.Properties.Length;
                    if(destinationFileLength != sourceFileLength)
                        throw new Exception(string.Format("The copied file {0} does not have the same size as the source file.",destinationFileBlobName));
    
                }
    
                //
                //Publish the asset:
                var destinationAssetFile = assetToBeProcessed.AssetFiles.Create(destinationFileBlobName);
                destinationAssetFile.IsPrimary = true;
                //destinationAssetFile.ContentFileSize = destinationFileLength; //setter only available in SDK > v2.0.0.5
                destinationAssetFile.Update();
    
                assetToBeProcessed = RefreshAsset(assetToBeProcessed, _context);
    
                //
                //At this point, you can create a job using your asset.
                // ...
                Console.WriteLine(String.Format("You are ready to use your asset with Id: {0} Name: {1} FileCount: {2}", assetToBeProcessed.Id, assetToBeProcessed.Name, assetToBeProcessed.AssetFiles.Count()));
                //
            }
    
            //Where:
    
            public static IAsset RefreshAsset(IAsset asset, CloudMediaContext context)
            {
                asset = context.Assets.Where(a => a.Id == asset.Id).FirstOrDefault();
                return asset;
            }

    Thursday, November 29, 2012 6:40 PM
    Owner
  • Dears

              In the above code there is one code segment that is "destinationFileBlob.CopyFromBlob(sourceFileBlob);". Through my testng, I find the destinationFileBlob and sourceFileBlob must be the same type of CloudBlob. But my blob type is the type of CloudPageBlob. I try to transfer my blob to Cloudblob as below. But I think it is inefficient .So is there another way to handle it simply?      

                  //sourceFileBlob is CloudPageBlob

                    BlobStream sourceStream = sourceFileBlob.OpenRead();
                    byte[] buffer = new byte[(int)transcodingInfo.BlobInfo.FileSize];
                    sourceStream.Read(buffer, 0, (int)transcodingInfo.BlobInfo.FileSize);
                    CloudBlob destinationFileBlob = destinationContainer.GetBlobReference(blobName);
                    destinationFileBlob.UploadByteArray(buffer);
                    //Check destination blob:         
                    destinationFileBlob.FetchAttributes();

    Friday, January 11, 2013 3:16 AM
  • Just to round out this thread, we just published a new article on Copying Blobs on MSDN.

    How-to: Copy an Existing Blob into a Media Services Asset

    http://msdn.microsoft.com/en-us/library/jj933290.aspx

    Thursday, February 07, 2013 8:57 PM
    Owner
  • Hi John,

    When I use the code from the How-to, the asset gets created but the size is shown as 0. As a result if I create a Streaming URL I see the URL in the Azure Portal (Published URL) but it fails to render on any client.

    However, if I click on the 'Sync Metadata' button in Azure Portal, then after a few seconds the size of the asset gets updated from 0 to the correct size and then the clients start working.

    Wondering if there is an API to do the equivalent of 'Sync Metadata'?

    Thanks and Regards,

    Sumit.

    Thursday, August 01, 2013 11:23 PM
  • Never mind, found the fix. I was missing the lines of code to set the primary asset file and call Update() on the asset file. Once I did that, the clients started working even without hitting Sync Metadata. Even though the portal continues to show size as 0 now!
    Thursday, August 01, 2013 11:40 PM