locked
Random "NotFound" error returned from Azure Storage RRS feed

  • Question

  • Hello,

    We have an app in production that is experiencing some worrisome Azure-related problems. This is a feature that was previously working and then now, without any site updates, is failing sometimes with the unhelpful error "NotFound" returned from Azure. The server task that is performing these Azure storage requests is basically just copying blobs from one container to another.

    Our users have to keep attempting the operation, and then after as many as 10 tries it will finally succeed without the error. It's elusive and the error is non-descriptive, so it's incredibly difficult to diagnose.

    Anything that I should look into after getting an error like this?

    Thanks for any help,

    -Sam

    Friday, November 10, 2017 6:44 PM

All replies

  • Are you sure your program is catching all errors? You could include code to check for problems, such as verifying that a file and/or folder exists (when relevant).

    You might investigate solutions for reporting errors. You should at least ensure you have the location (source code line number) of the error. I assume you don't have that since you don't show the relevant source code or any details of what is happening.



    Sam Hobbs
    SimpleSamples.Info

    Friday, November 10, 2017 6:50 PM
  • I do need to narrow down the line, yes. But I am catching the error because it is being logged in our server logs. I will try to narrow it down. The thing is, this didn't use to happen and now, without any code changes, it's happening, so it seems unlikely to be a code issue.
    Friday, November 10, 2017 11:38 PM
  • so it seems unlikely to be a code issue.
    I've heard that before. It could be a combination of external and internal. It could be that the program could be improved to be unaffected by the external conditions. The thing we don't know is if the problem could have been anticipated.


    Sam Hobbs
    SimpleSamples.Info

    • Proposed as answer by Md Shihab Sunday, November 12, 2017 2:50 AM
    Saturday, November 11, 2017 12:06 AM
  • Gotchya. I'll take a look and see if I can narrow it down. Thanks!
    Saturday, November 11, 2017 12:09 AM
  • Threw some console logs in there and restarted the server and now of course it works fine...do I have to just wait until this happens again?
    Sunday, November 12, 2017 7:45 AM
  • Nevermind, I've managed to see the error happen again and I've narrowed it down! The line that is making it fail:

    return blobSvc.getBlobToStreamAsync(container, blobId, fs.createWriteStream(path));

    Now, it's worth mentioning that I promisified the Azure storage module in node, that's why it has "Async" at the end. The point is that I am catching any errors in a .catch after the returned promise.

    I know that the blob I am trying to stream exists, because it was created just before this request with the following:

            try {
                blobSvc.createBlockBlobFromLocalFile(container, blobName, fileName, function(err, result, response){
                    try {
                        if (err) return reject(err);
                        if (!response.isSuccessful) return reject(new Error('Failed to create blob.'));
    
                        return resolve(config.BLOB_BASE_URL + '/' + container + '/' + blobName);
                    } catch (err) {
                        return reject(err);
                    }
                })
            } catch (err) {
                return reject(err);
            }

    As you can see, I'm catching all possible errors. Furthermore, sometimes the error is only a "NotFound" error, other times it includes a second error "the specified container does not exist" (something like that), even though I know it exists because I just wrote a couple files into it (second code block above). The fact that this happens intermittently, and that Azure only seems to have started doing this recently, tells me that something has changed in the storage account system in Azure.

    EDIT: one final thing to note is that I couldn't reproduce this error on our staging site after I threw in a bunch of console logs, it only started happening when I pushed the update to the production site. Both staging and production are using the same database and code, so the only difference is the URL of the site.
    Sunday, November 12, 2017 9:27 AM
  • @Sam12345678901

    I suggest you raise a technical support ticket to troubleshoot this further as we’d have to take a closer look at it since this appears to be occurring intermittently.

    -----------------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.

    Thursday, November 16, 2017 4:43 AM