locked
Unable to set Proxy for CloudBlobClient RRS feed

  • Question

  • Hi,

    we have a scenario that we are uploading large files (more than 12 GB) into azure blob storage from our java application. 

    For security purpose when ever our application calling out side application like azure cloud we have to set proxy.

    As of now we are setting proxy at system properties level and it is working as expected.

    But we have a problem for other internal consumer services if we use system properties level and other services all failing. 

    Then we tried to set proxy at specific client level , we tried two way (Restful , using SDK's) and getting following problem.

    Using HTTP Rest 

    ==============    

                    String requestMethod = "PUT";
           String containerName = "testBlob";
           String blobName="Sample";
           String accountName="storageaccount";
           String accountKey="XXXXXXXXXXXXXXXX";


           String urlPath = containerName+"/"+ blobName; 
           String storageServiceVersion = "2013-08-15"; 
           SimpleDateFormat fmt = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss");  
           fmt.setTimeZone(TimeZone.getTimeZone("GMT"));  
           String date = fmt.format(Calendar.getInstance().getTime()) + " GMT";  
           String blobType = "BlockBlob";

           URI uri =null;

           try{

    uri = new URI("http://"+ accountName + ".blob.core.windows.net/" + urlPath);
    final PutMethod postMethod = new PutMethod(uri.toString());        
               HttpClient httpClient = new HttpClient();  
               final HostConfiguration hostConfiguration = new HostConfiguration();
    hostConfiguration.setProxy("hostname",20);            
    httpClient.setHostConfiguration(hostConfiguration);

              String canonicalizedHeaders = "x-ms-blob-type:"+blobType+"\nx-ms-date:"+date+"\nx-ms-version:"+storageServiceVersion;  
              String myAccount="/"+ accountName +"/"+urlPath;

              String stringToSign = requestMethod+"\n\n\n"+tempFile.length()+"\n\n"+"application/octet-stream"+"\n\n\n\n\n\n\n"+canonicalizedHeaders+"\n"+myAccount; 


              String authorizationHeader = createAuthorizationHeader(stringToSign,accountName,accountKey);


      postMethod.addRequestHeader("x-ms-blob-type", blobType);  
      postMethod.addRequestHeader("x-ms-date", date);  
      postMethod.addRequestHeader("x-ms-version", storageServiceVersion);
              postMethod.addRequestHeader("Authorization", authorizationHeader); 

              postMethod.addRequestHeader("Content-Length", String.valueOf(tempFile.length()));

             FileRequestEntity requestEntity = new FileRequestEntity(tempFile,"application/octet-stream");
             postMethod.setRequestEntity(requestEntity);

             final int statusCode = httpClient.executeMethod(postMethod);  

             if (statusCode != 201 && statusCode != 200) {

    String errrCode = "statusCode=====>"+statusCode;

    }

           }catch(Exception ex)
           {
    ex.printStackTrace();
           }

    Result :

    we can able to set proxy at specific client level but we are failing to upload large files, even we are failing 500 MB also.

    USING SDK's

    =========

                          // get the account 
    CloudStorageAccount storageAccount = CloudStorageAccount.parse(storageConnectionString);
    // create the blob client
    CloudBlobClient blobClient = storageAccount.createCloudBlobClient();
    // get the container reference
    CloudBlobContainer container = blobClient.getContainerReference("testBlob");
    CloudBlockBlob blob = container.getBlockBlobReference("testfile");
    blob.upload(new FileInputStream(tempFile), c);

    Result :

    we can able to succeed for uploading large files. but the problem with setting proxy at system properties level.

    Can anyone help on this is there any process to set proxy at client level ?


    • Moved by Sheethal J S Tuesday, July 11, 2017 4:29 AM More related to Azure Storage
    Monday, July 10, 2017 7:02 AM

Answers

  • What is the size of files that you could successfully upload with the client proxy configured? Could you share the exact error message that you get when the large file upload fails?

    -----------------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.

    Tuesday, July 11, 2017 5:47 AM

All replies

  • What is the size of files that you could successfully upload with the client proxy configured? Could you share the exact error message that you get when the large file upload fails?

    -----------------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.

    Tuesday, July 11, 2017 5:47 AM
  • Hi Shihab,

    while we are trying with REST process we are not getting any error in logs but its trying long time and stopped process. we got succeed up to 30MB using REST process.

    we can able upload using 2nd process but getting problem with  with proxy. 

    Tuesday, July 11, 2017 8:19 AM
  • Using the Java client, you can pass in an “OperationContext” object when you call ‘upload’.  This object has a get/setProxy() method, which should cause the call to use the input proxy.  You can also use get/setDefaultProxy() to set a static proxy, for all calls through the Java Storage Client. Could you try that, if not done already?

    -----------------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.

    Tuesday, July 18, 2017 3:57 AM