locked
404 error while updating set of documents manually RRS feed

  • Question

  • Hello,

    On our project we upload documents from client code (using 'MergeOrUpload' action): single document at a time. Later set of documents can be updated (using 'Merge' action). The problem is that sometimes we are receiving 404 error when performing 'Update' on set of documents (for some documents in batch). At the same time ALL 'Upload' operations don't report on any issue.

    This is happening on production only (with S1 tier, 3 replicas and 2 partitions) and we can't understand why (we cannot reproduce it locally).

    Maybe someone have any thoughts on this? Did someone experienced this before?

    Thanks in advance!






    Monday, June 4, 2018 11:25 AM

Answers

  • Hi Siarhei,

    Please feel free to e-mail me directly so I can help you look into your case specifically.  My e-mail is cmacdo@microsoft.com

    Thanks,

    Carey

    • Marked as answer by Siarhei Machel Tuesday, October 30, 2018 2:10 PM
    Wednesday, June 6, 2018 6:36 PM

All replies

  • Hi Siarhei,

    Is it possible that you are attempting to update the key of the document as part of Merge action?  A status code of 404 on Merge means that the given key does not exist in the index.  In the case of updating the key, you will need to Upload the new document and Delete the old version as Merge will not work properly for a change in key.

    Also, please check the status code of your initial MergeOrUpload action as well.  You could be getting a response of 207, which means that some documents may be succeeding and others may not, which would explain why you see issues with some and not others in the later Merge operation.

    Please refer to the documentation for these operations for more details.  We are actively working on adding some more specific details for individual error codes that you should be able to see soon.

    https://docs.microsoft.com/en-us/rest/api/searchservice/addupdate-or-delete-documents

    Thanks,

    Carey MacDonald


    Tuesday, June 5, 2018 4:23 PM
  • Hi Carey,

    Thanks for the response.

    Regarding first question: no, we are not updating document keys at all.

    Yesterday we requested Help Desk to check whether these documents exist in the index: we were very surprised because most of the documents for which we saw 404 errors DO exist (but still there are few absent). I 100% sure that Merge action is being performed on the same document's keys (both upload and merge use the same piece of code for that).

    Also, initial upload is being performed using Retry mechanism so that in case we get failed items in a batch it reties index request up to 20 times (for failed items only). And we don't see any log records complaining about initial upload.

    Is it possible that on high load hours sometimes indexing itself (meaning background process) can delay significantly (saying delay i mean time gap between search service accepts request and time when document got indexed eventually)?

    Thanks

    Wednesday, June 6, 2018 8:19 AM
  • Could you please also tell me how errors are being handled in background indexing process? 

    Indexing workflow is the following:

    1. We send an API call to search service (with batch of documents)

    2. Then azure search service accepts (or rejects) the request

    3. Starts background task to index documents

    4. And finally returns a response.

    According to this workflow we can get a response back before all documents are actually indexed (and this is known/documented behavior).

    My question is: as far as search service can send response before it starts indexing how would we know if something failed during indexing phase? Does search service logs any information about failed or successful index operations (monitoring shows logs for API calls only)?

    I'm asking this because few month ago we faced an issue when azure search was silently skipping some indexing requests (at the same time every response it used to return was successful: 200), but total number of indexed documents was less than number of documents in requests we send. It was applicable to old version of search service only (saying old i mean the one that was deployed before announcement of some updates in Jan). On new version we couldn't reproduce that. 

    Wednesday, June 6, 2018 10:39 AM
  • Hi Siarhei,

    Please feel free to e-mail me directly so I can help you look into your case specifically.  My e-mail is cmacdo@microsoft.com

    Thanks,

    Carey

    • Marked as answer by Siarhei Machel Tuesday, October 30, 2018 2:10 PM
    Wednesday, June 6, 2018 6:36 PM