none
411 error when invoking ADLS Gen2 PUT rest call to create directory RRS feed

  • Question

  • When I am trying to create directory in ADLS Gen2 with the below code in Java

                  
    String accountName = "xxx";
    String accessKey = "yyy";		
    String uri = "https://"+accountName+".dfs.core.windows.net/datalakefs1/Test2";		
    
    Client client = ClientBuilder.newClient().property(HttpUrlConnectorProvider.SET_METHOD_WORKAROUND, true);
    		client.property(ClientProperties.SUPPRESS_HTTP_COMPLIANCE_VALIDATION, true);
    WebTarget target = client.target(uri);
    		
    String gmDate1 = getStorageDate();
    
    String StringToSign = "PUT\n"
    		+ "\n" // content encoding
    		+ "\n" // content language
    		+ "0\n"// content length	
    		+ "\n" // content md5
    		+ "\n" // content type
    		+ "\n" // date
    		+ "\n" // if modified since
    		+ "\n" // if match
    		+ "\n" // if none match
    		+ "\n" // if unmodified since
    		+ "\n" // range
    		+ "x-ms-date:" + gmDate1 + "\n" + "x-ms-version:2018-11-09" + "\n" // headers
    		+ "/xxx/datalakefs1/Test2\nresource:directory"; // resources
    		
    String	authorizationStringToHash = getAuthenticationString(accountName, accessKey, StringToSign);
    		
    Response response  = target.queryParam("resource", "directory")
    				.request().header("Authorization", authorizationStringToHash)
    				.header("content-length", "0")
    				.header("x-ms-date", gmDate1)
    				.header("x-ms-version", "2018-11-09")
    				.method("PUT");

    I got the below error : 

    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
    <HTML><HEAD><TITLE>Length Required</TITLE>
    <META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
    <BODY><h2>Length Required</h2>
    <hr><p>HTTP Error 411. The request must be chunked or have a content length.</p>
    </BODY></HTML>

    I am using below methods to get x-ms-date and authorization value :

    public static String getStorageDate() {
    		Date d = new Date(System.currentTimeMillis());
    		SimpleDateFormat f = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss zzz");
    		f.setTimeZone(TimeZone.getTimeZone("GMT"));
    		String dateString = f.format(d);
    		return dateString;
    	}
    
    	private static String getAuthenticationString(String accountName, String accessKey, String stringToSign)
    			throws Exception {
    		Mac mac = Mac.getInstance("HmacSHA256");
    		mac.init(new SecretKeySpec(Base64.decodeBase64(accessKey), "HmacSHA256"));
    		String authKey = new String(Base64.encodeBase64(mac.doFinal(stringToSign.getBytes("UTF-8"))));
    		String auth = "SharedKey " + accountName + ":" + authKey;
    		return auth;
    	}

    Please let me know the issue with this code.

    Thanks,

    Ramya


    • Edited by Ramya_K Tuesday, April 23, 2019 9:49 AM
    Tuesday, April 23, 2019 9:47 AM

Answers

  • Issue is resolved after removing the content-length (from the header and the signature) and setting the content-type.

    We cannot set the content-length as Jersey automatically sets the content-length header based on the payload data.

    Below is the updated code: 

    String StringToSign = "PUT\n"
    		+ "\n" // content encoding
    		+ "\n" // content language
    		+ "\n"// content length	
    		+ "\n" // content md5
    		+ "text/plain\n" // content type
    		+ "\n" // date
    		+ "\n" // if modified since
    		+ "\n" // if match
    		+ "\n" // if none match
    		+ "\n" // if unmodified since
    		+ "\n" // range
    		+ "x-ms-date:" + gmDate1 + "\n" + "x-ms-version:2018-11-09" + "\n" // headers
    		+ "/xxx/datalakefs1/Test2\nresource:directory"; // resources
    		
    String	authorizationStringToHash = getAuthenticationString(accountName, accessKey, StringToSign);
    
    Response response  = target.queryParam("resource", "directory")			
    				.request().header("Authorization", authorizationStringToHash)
    				.header("content-type","text/plain")
    				.header("x-ms-date", gmDate1)
    				.header("x-ms-version", "2018-11-09")								
    				.put(empty);
    

    • Marked as answer by Ramya_K Friday, April 26, 2019 6:48 AM
    Friday, April 26, 2019 6:30 AM

All replies

  • Hi Ramya,

    The URI should be in the format :

    https://<storage account name>.dfs.core.windows.net/<file system name>/<name of directory to be created>?resource=directory.

    Please try this out and let us know if it works. Else, we can continue to dive deeper into the issue.


    MSDN

    Wednesday, April 24, 2019 11:00 AM
    Moderator
  • Hi ChiragMishra,

    The URI is in the same format as you have mentioned.

    In the below line of code, datalakefs1 is the file system and Test2 is the directory to be created.

    String uri = "https://"+accountName+".dfs.core.windows.net/datalakefs1/Test2";	


    I have specified the query parameters as below :

    target.queryParam("resource", "directory")

    It is working in Postman, when I execute the PUT request with the same headers and query params.


    Thanks,

    Ramya

    Thursday, April 25, 2019 11:11 AM
  • Issue is resolved after removing the content-length (from the header and the signature) and setting the content-type.

    We cannot set the content-length as Jersey automatically sets the content-length header based on the payload data.

    Below is the updated code: 

    String StringToSign = "PUT\n"
    		+ "\n" // content encoding
    		+ "\n" // content language
    		+ "\n"// content length	
    		+ "\n" // content md5
    		+ "text/plain\n" // content type
    		+ "\n" // date
    		+ "\n" // if modified since
    		+ "\n" // if match
    		+ "\n" // if none match
    		+ "\n" // if unmodified since
    		+ "\n" // range
    		+ "x-ms-date:" + gmDate1 + "\n" + "x-ms-version:2018-11-09" + "\n" // headers
    		+ "/xxx/datalakefs1/Test2\nresource:directory"; // resources
    		
    String	authorizationStringToHash = getAuthenticationString(accountName, accessKey, StringToSign);
    
    Response response  = target.queryParam("resource", "directory")			
    				.request().header("Authorization", authorizationStringToHash)
    				.header("content-type","text/plain")
    				.header("x-ms-date", gmDate1)
    				.header("x-ms-version", "2018-11-09")								
    				.put(empty);
    

    • Marked as answer by Ramya_K Friday, April 26, 2019 6:48 AM
    Friday, April 26, 2019 6:30 AM
  • Hi Ramya,

    Glad to hear that the problem was resolved. Also, thanks for sharing the resoultion. It will be helpful for the community at large.


    MSDN

    Monday, April 29, 2019 9:24 AM
    Moderator
  • Hello Ramya, 

    I am trying to create a file system on ADLS GEN 2 account with hierachical namespace enabled.  Only difference is endpoint is different.  I have 2 scripts - Powershell that works just fine and 2nd one in Bash (That does not work at all)

    Would appreciate any feedback. 

    Usage:

    ./create-filesystem.sh storageaccountname filesystemname accesskey

    Error:

    curl: (6) Could not resolve host: PUT

    {"error":{"code":"AuthenticationFailed","message":"Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:e422daa5-901f-00d8-0231-54a8cc000000\nTime:2019-08-16T12:54:36.0101349Z"}}https://storageaccountname.dfs.core.windows.net/filesystem?resource=filesystem

    #!/usr/bin/env bash
    STORAGE_ACCOUNT_NAME=${1}
    FILESYSTEM_NAME=${2}
    ACCESS_KEY=${3}
    request_date=$(TZ=GMT date "+%a, %d %h %Y %H:%M:%S %Z")
    #request_date=$(date +%Y-%m-%d)
    storage_service_version="2018-11-09"
    # HTTP Request headers
    x_ms_date_h="x-ms-date:$request_date"
    x_ms_version_h="x-ms-version:$storage_service_version"
    n="\n"
    method="PUT"
    string_to_sign+="$method\n\n\n\n\n\n\n\n\n\n\n\n$x_ms_date_h\n$x_ms_version_h\n/$STORAGE_ACCOUNT_NAME/$FILESYSTEM_NAME\nresource:filesystem"
    # Decode the Base64 encoded access key, convert to Hex.
    shared_key="$(echo -n $ACCESS_KEY | base64 -d -w0 | xxd -p -c256)"
    # Create the HMAC signature for the Authorization header
    signed_signature=$(printf "$string_to_sign" | iconv -t utf-8 | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$shared_key" -binary | base64 -w0)
    auth_header="Authorization: SharedKey ${STORAGE_ACCOUNT_NAME}:$signed_signature"
    uri="https://$STORAGE_ACCOUNT_NAME.dfs.core.windows.net/$FILESYSTEM_NAME?resource=filesystem"
    curl -d -X $method -H "$x_ms_date_h" -H "$x_ms_version_h" -H "$auth_header" $uri
    if [ $? -eq 0 ]; then
    echo $uri
    exit 0;
    fi;
    exit 1

    Friday, August 16, 2019 4:23 PM