none
APPEND failed with error 0x83090a16 (Internal server error.) RRS feed

  • Question

  • Hi, 

    What exactly does the error "APPEND failed with error 0x83090a16 (Internal server error.)" mean?

    I'm using the Java ADLS SDK and sometimes i get this error when appending to a file from multiple threads. It appears that ADLS locks the file for writing just like normal File Systems do. Is this reasoning correct?

    I've manage to largely fix the problem by adding something like a thread id to the file name being written to guarantee that only a specific thread appends to that file and avoid the concurrency issues. The problem is that from time to time i still get that error even without multiple threads appending to the same file. 

    Can you guys describe what exactly does that error mean and the best way to avoid it.

    Thanks

    Thursday, February 9, 2017 6:42 PM

All replies

  • This could happen if you have two streams writing to the same file (same thread or different threads – this is not a threading issue). Are you doing multiple createFile or getAppendStream calls pointing to the same file, and then appending to the resultant streams?  If this is not the case, can you please give more details?  Thanks

    Wednesday, February 15, 2017 6:35 PM
  • I'm receiving messages at high speed from an Apache Kafka Cluster and i'm appending batches of those messages to a file in ADLS. Each batch has +/- 1000 messages and each message is around 1KB.

    What i'm doing is: for each batch, i get an append stream, write the batch and then close the stream. When i have multiple processes doing this simultaneous, i get that error.

    My workaround to avoid the error was to create a file per process and avoid having multiple processes writing to the same file. My problem is that i still get some errors if the batch size is too large.  

    Note that i'm not reusing the stream returned from getAppendStream for multiple writes, i'm creating an append stream for each write. (and closing it after the write)

    Thanks

    Thursday, February 16, 2017 11:13 AM
  • Just noticed this, not sure if you already worked around your problem. For performance, it is better to use the same stream within the same thread (rather than closing and reopening every time, which results in multiple server roundtrips).

    Is it possible that on a large batch this is happening if the processes crash? One way I can see this happen is if a process crashes and a replacement process tries to open the file with the same name within 10 mins - in this case, the old lease would still be held on the file, preventing a getAppendStream() from succeeding.

    Friday, April 28, 2017 6:56 PM