none
Corruption issues with FSLogix and Server 2019 utilizing OneDrive Files on Demand RRS feed

  • Question

  • I have a case open with Microsoft.  

    FSLogix 2.9.7117.27413 / OneDrive 19.123.0624.0005 / Server 2019 utilizing Files on Demand.

      When sync'ing a SharePoint library, the files will turn into 0KB files, then sync back up into SharePoint overwriting all of the good files. I can recreate this issue. It will also delete files and put them in the recycle bin. This issue does NOT occur on a basic Server 2019 server without FSLogix. I have a test environment setup where I am working this issue.

    -----------------------------------------------------------

    The first case, the guy took my support logs and didn't respond to the case for 5 days, even after I entered multiple updates asking for a status.  

    I then "escalated" the case with my MS Rep, who had me open up another case (Sev B) with ASfP (Premier) support through the Azure portal.  I entered all the information, and the support logs into the case.  1:57 minutes later, the engineer asked me to run the support tool and attach the logs (which are already attached to the case).  I have received no response yes, 24 hours later.  

    We were forced to pay $15k for this "Premier" support, and this is how i'm being treated.

    Is that what i'm supposed to expect from Microsoft Support for FSLogix?  



    Tuesday, August 20, 2019 3:12 PM

All replies

  • Hello,

    I'm sorry to hear that you haven't had a good support experience. 
    Can you please have your TAM send me an IM in MS Teams and send me your SR# so that I can see what I can do to help get this issue resolved for you? Thanks!

    According to the FSLogix Support SLA:

      • FSLogix support is not fully integrated into the regular Microsoft support work flows yet after its recent acquisition and currently only offers support in global English with daily e-mail responses (or call backs) as needed. We have a goal of meeting standard Microsoft SLAs for confirmed service down issues, but for all other issue types we are still working to the FSLogix SLA. Thanks for your patience! 

      - Micah Adamson

    Tuesday, August 20, 2019 5:16 PM
    Owner
  • Hello,
    I think I found your two tickets from August 14 and 19 regarding:
    FSLogix 2.9.7117.27413 / OneDrive 19.123.0624.0005 / Server 2019 utilizing Files on Demand
    I'm working with the two different assigned agents to merge these tickets and contact you as soon as possible.
    - Micah Adamson
    Tuesday, August 20, 2019 6:01 PM
    Owner
  • Whatever you did worked, because Cassondra has emailed me.  I have worked with her before when they were just FSLogix, and she is great.  Thanks!  

    Tuesday, August 20, 2019 7:07 PM
  • Glad to hear it! I'm going to merge your two tickets into one and make sure Cassondra is assiting you with that case. Thanks!
    Tuesday, August 20, 2019 7:08 PM
    Owner
  • I can't find any actual documentation.  Is my configuration even supported?

    Files on Demand on Server 2019.....  ?  Anyone know?  

    Wednesday, August 21, 2019 6:13 PM
  • According to: the FSLogix Release Notes it should be since...
    FSLogix Apps 2.9.1 (2.9.6764.27422)
    - Full support for Files On Demand feature of OneDrive

    Do you use FSLogix Profile Containers or are you using a different profile solution with FSLogix Office Containers?

    It would be very helpful for the FSLogix DEV team to know if this problem is only effecting Office Containers or also Profile Containers, since this looks like it may be a bug in either FSLogix or OneDrive. Thanks!

    - Micah Adamson

    Windows Server 2019 should be fully supported for FSLogix and Office 365 now too according to:
    https://rcpmag.com/articles/2019/07/10/windows-server-2019-support-office-365-proplus.aspx


    Wednesday, August 21, 2019 6:25 PM
    Owner
  • According to: the FSLogix Release Notes it should be since...
    FSLogix Apps 2.9.1 (2.9.6764.27422)
    - Full support for Files On Demand feature of OneDrive

    Do you use FSLogix Profile Containers or are you using a different profile solution with FSLogix Office Containers?

    It would be very helpful for the FSLogix DEV team to know if this problem is only effecting Office Containers or also Profile Containers, since this looks like it may be a bug in either FSLogix or OneDrive. Thanks!

    - Micah Adamson

    Great, thanks.  The ticket is with DEV right now.  I use both FSLogix Profile containers and O365 containers together in this environment.  I believe they are aware of this, however, I will make sure they know.  
    Wednesday, August 21, 2019 6:29 PM
  • FYI: I wanted to confirm that there is a known issue that is being addressed by the FSLogix and OneDrive DEV teams where data can become corrupt if you use this specific configuration. We'll post more information once it becomes available. Thanks for your patience.


    Wednesday, August 28, 2019 5:43 PM
    Owner
  • Exist an Workaround?

    giovanni.parrilla@avectris.ch

    Tuesday, September 3, 2019 12:58 PM
  • Hello Everyone,

     

    The FSLogix and OneDrive teams are still working on a solution for this issue. Until then, I wanted to post some information I found about OneDrive Files on Demand support with Windows versions, even without FSLogix involved.

     

    Windows Server 2016 does NOT support OneDrive Files on Demand:

     

    Windows Server 2019 supports OneDrive Files on Demand:

     

    Windows 10 Fall Creator's Update (or newer) supports OneDrive Files on Demand:

     

    It looks like only the latest versions of Windows 10 and Windows Server will work with OneDrive files on Demand. So, once this issue is resolved, FSLogix will only support OneDrive Files on Demand with the latest versions of Windows.

     

    Thanks again for your patience as we work to get this issue resolved.

    -Micah Adamson



    Wednesday, September 4, 2019 5:45 PM
    Owner
  • FYI: I wanted to confirm that there is a known issue that is being addressed by the FSLogix and OneDrive DEV teams where data can become corrupt if you use this specific configuration. We'll post more information once it becomes available. Thanks for your patience.


    Looking at the latest release of FSL 1909 I don't see reference to the above in the list of fixes.

    Is this still an issue if Files On Demand is enabled on Server 2019 and  the OneDrive cache isn't redirected to the container? AKA IncludeOneDrive is set to 0 in the registry?

    Saturday, October 5, 2019 1:49 PM
  • I am very disappointed in how Microsoft has handled the fslogix merge. I have nermous individuals in slack that help support fslogix in customers environment. The lack of response from fslogix unacceptable. I even had a meeting with the fslogix support engineer a manager all honesty I feel bad for them. I also feel like they weren’t ready for this, and the sad this the customers are suffering the most. I As a customer I had to deal thai as well. The lack up support from what is supposed to be a expert is disappointing. Microsoft do you understand how your impacting us? Is making us look terrible. Stop saying you're sorry and start supporting you customers. Its not our fault you guys dont have your stuff together. Its appeara updates are released with no Verison notes , no announcement. Are we the beta testers? Fix this soon please.
    • Edited by davisray1983 Saturday, February 29, 2020 9:05 PM
    Saturday, February 29, 2020 9:02 PM
  • I afraid I must echo your sentiment. We utilize sharepoint sync on our devices. In our Citrix environment we utilize FSlogix with Files on Demand (Server 2019). Onedrive seems to work well, but twice now a user has synced a sharepoint online library and when they log in a second time all kinds of sharepoint files are corrupted (files are renamed, deleted, etc.). They are not zeroed out like a previous issue, but they simply become corrupted, being renamed to a completely different file name and extension so the file handlers don't know how to handle them.  The worst part of this is there is no version history added so we have to go to external backups to restore whole document libraries.  I have had a ticket in the system, severity A since February 20th.  4 times i've called they keep 'escalating it'. Today they told me its a queue where engineers only can 'pick it up' and they don't have an ETA despite this being a premium ticket with a 2 hour response time.  Microsoft's customers rely on reliable sharepoint online availability in these types of environments, and FSLogix is a good product when working, but this isn't sustainable.  They need to find a way to get its customers the help they need. Restoring files libraries isn't something we can tolerate for a few months until they can get it sorted out.
    <style></style>
    • Edited by GTPBBC Monday, March 2, 2020 2:27 PM
    Monday, March 2, 2020 2:27 PM
  • Micah,

    Any update on this issue? We are experiencing the same issue multiple times a week after implementing FSLogix for one of our customers.

    Config:

    OS: RDS farm on Windows Server 2019 Standard (17763.1131)
    OneDrive: 19.232.1124.0010 (per machine installation, ie in Program Files)
    IncludeOneDrive is set to 1

    I've seen this happening at other customers too. It mostly happens with large SharePoint file libraries and even without using FSLogix. I discussed this with the SharePoint/Office 365 Team [Ticket #:13155103] about a year ago. The issue was never resolved and we ended up restoring the Sharepoint site to a new one. During that year we saw this happening one more time, did not bother to create a ticket. Just restored the lost data to a new sharepoint site.

    It does seem that FSLogix intensifies the issue. It happens a lot more often. We've seen this happening like once or twice a year without FSLogix. Different customers, different tenants, no correlation. With FSLogix we see this happening multiple times a week in a single tenant.

    This week, we had to recover 2 sites from backup because OneDrive is going haywire. In SharePoint we can see the following things happening in a rapid pace:

    - Files being renamed (even with file extensions all together).
    - Files being removed
    - Files being moved to other locations

    End result?

    - Files get lost
    - Folders become empty
    - All OneDrive clients who sync the affected Sharepoint site will go haywire a few minutes later, so you will have to remove all members from sharepoint in order to calm things down. 
    - User clicks on FILE A, sees FILE B (And because of this a lot of files won't open, because the file extension does change mostly with it. Like FinanceReport2020.pdf gets renamed to InternalPhoneList.xlsx. That's a weird name change for a user to make. And not possible in this case because we're talking about 100's of these changes per hour. Fun note: You can rename affected files back to .xlsx, .pdf, .docx, etc.. Just keep trying file extensions until you have found the right one and a corrupted document will open.)

    I also think it has something to do with latency, but that's an assumption. Something like OneDrive starting to sync but the storage is not ready or slow. In almost all cases the user who triggers the process reported some sign in issues that day.

    We have seen the same behavior as discussed here, even without FSLogix. I think OneDrive is the main reason for this issue, nonetheless, combining it with FSLogix seems to really intensify the problem. As said by GTPBBC, restoring all the time is not something we can manage for a few months. Is the OneDrive support team included and informed about this case? 

    Could you provide a status update on this issue?

    Thanks,

    Monday, March 30, 2020 6:58 PM
  • Hi Micah

    Do you have an update on this case?

    Or maybe a bug id / ticket I'd we can use when contacting support? 

    Thursday, April 2, 2020 6:45 PM
  • Hi,

    Ran into the exact same problem today.

    Luckily we just finished migrating our file server data to SharePoint, so data loss is minimal, had to restore data from the old file-server and setup file shares instead of using SharePoint.

    Could be nice with some warnings about this from Microsoft regarding SharePoint and OneDrive on Terminal services on server 2019.

    Do any of you have issues with ordinary OneDrive and Server 2019 or is it only when you sync SharePoint data ?

    Thanks for this information, it really helped pinpointing the problem


    Thursday, April 2, 2020 10:01 PM
  • Hi,

    So far for us we have not seen this issue return since last year after updating to 2.9.7237.48865

    The issue was brutal before: 0 byte files, modify date set to 1980, files synced over the top of other files e.g. Excel file saved over PDF file...it was a mess.

    1. We updated to latest FSLogix release.

    2. We also decided to place OneDrive in the Profile container and only Outlook in the ODFC container (found this more stable even when the issue started occurring)

    Been running user workloads since December, with users using synced SharePoint files and OneDrive, and so-far this particular issue has not come back... just need to resolve all the other more minor issues now.

    Tuesday, April 7, 2020 8:15 AM
  • "2. We also decided to place OneDrive in the Profile container and only Outlook in the ODFC container (found this more stable even when the issue started occurring)"

    That might be a solution! I think our situation is not that different, besides your FSLogix config.

    - We use the same version of FSLogix ie 2.9.7237.48865
    - Both with Profile and Office 365 containers.
    - Regular UPD's in the session collection are disabled.
    - OneDrive is in the Office 365 container.
    - We have seen this only happening for Sharepoint syncs (so far).

    Might I ask to share your config swelzim? Profile Container, Office 365 Container and how Cloud Cache is enabled? I would like to compare it with ours and create the exact same config as you. I think we might be onto something here..


    Tuesday, April 7, 2020 4:32 PM
  • We have more or less the same problem. Is there allready a solution given by microsoft?

    This makes fslogix & files on demand useless for us.


    Wednesday, May 13, 2020 10:44 AM
  • Same problem here, updated to the latest version of the FSLogix agent on Server 2019 but the problem still exists. What to do?
    Friday, May 15, 2020 3:58 PM
  • We are using the same configuration and also having these kind of issues. On top of that, we frequently see the OneDrive Files on Demand option just gone missing.. resulting all synced sites will be fully downloaded. 

    That just simply kills our storage capacity.. 

    Saturday, May 16, 2020 8:56 AM
  • This issue was resolved in FSLogix_Apps_2.9.7237.48865.

    "An issue with Robocopy was mitigated when using FSLogix containers where, in rare cases, OneDrive and SharePoint files would have 0 bytes with a 1980 modified date.  This issue predated the FSLogix 1909 release.  Files that appear to have 0 bytes and a 1980 date may be recovered by using the version history.  OneDrive Sync, with Files On-Demand, is only supported on FSLogix release 1909 HF_01 or later.   Internal ID# 23714858"

    https://social.msdn.microsoft.com/Forums/en-US/dcd86de9-e092-49b4-b2a1-20e1943bbdc5/release-notes-for-fslogix-apps-1909-hf01-29723748865-hotfix?forum=FSLogix

    Tuesday, June 16, 2020 1:38 PM
  • Same issue for us. we had to recover more than 100gb of files. Ms confirmed the fslogix issue and they are working on a new release. But there aren’t official statemente from MS We deactivated the onedrive on the terminal servers 😠
    Friday, July 10, 2020 6:24 PM
  • Any new Information about this Issue? Or a Case where i can Reference to? Seeing also strange Issues with this Combination (Files deleted or corrupted, FOD Option in O4B gone). 
    • Proposed as answer by Moustique Thursday, August 6, 2020 3:15 PM
    • Unproposed as answer by Moustique Thursday, August 6, 2020 3:15 PM
    Thursday, July 30, 2020 5:55 AM
  • FYI: I wanted to confirm that there is a known issue that is being addressed by the FSLogix and OneDrive DEV teams where data can become corrupt if you use this specific configuration. We'll post more information once it becomes available. Thanks for your patience.


    This month of July we faced big issues with OneDrive files on demand, over servers RDSH 2019 wth Fslogix profiles and ODFC FSLogix for Office.
    This became a big mess. Those are the problems we faced :

    1.    Files deleted by OneDrive client
    2.    Files corrupted by OneDrive client
    3.    Files switched by other files due toOneDrive client (For example opening a PDF file « sellings2020.pdf » and the contents are from a completely different file, located in a completely different folder). Really strange.

    Seems Science-fiction, completely crazy.

    We noticed that some users of servers RDSH 2010 could not get OneDrive client with files on demand activated,
    we had to delete the FSLogix profile many times and create it again since it works out.

    We soon noticed that all this messy behavior came from OneDrive clients on the RDS servers, also local PC’s are using OneDrive.
    We removed the agent from the servers but the damage was already done.

    138 Gb of synced files, arround 83.000 files, sync turned for arround 18 days before we noticed the problem.
    The problem appeared progressively as users connected to the RDS servers.

    We spent 80 hours just to resotre corrupded, deleted or exchanged files.

    I can’t understand how you allow the installation of OneDrive with FSLogix when you already know that they are totally incompatible. It’s a real mess.

    Here there are the versions used :

    VMWare 6.7U3
    Microsoft Windows Serveur 2019 STD (1809 V. 17763.1339)
    Microsoft FSLogix Apps 2.9.7349.30108
    Microsoft Office Pro Plus 2019
    OneDrive 20.114.0607.0002

    If you need itw e can provide all our GPO configuration for FSLogix and OneDrive.

    Do you have a solution? Or do you plan to fix it in the near future.
    At least yo could announce officialy the incompatibility and inform people not to use OneDrive along with FSLogix profiles.

    This could save many headache and blank nights to our IT companions.
    Thursday, August 6, 2020 3:18 PM
  • Just want to add we are experiencing the same issue with OneDrive and the latest FSLogix client.  We're waiting on a fix before we can roll out our new Citrix environment.  We had to restore weeks of data from SharePoint because of OneDrive corruption.
    Friday, August 7, 2020 9:19 PM
  • Hi! Was the OneDrive client installed per user og per machine?

    Tuesday, August 11, 2020 10:16 AM
  • Per machine.  OneDriveSetup.exe /allusers
    Tuesday, August 11, 2020 4:44 PM
  • OK, we are running it per user for about 500 users in Win 2019 RDSH, and has not experienced any corruption yet.

    I don't think I have seen others report corruption with per user either..?

    We are running OneDrive in ODFC, and the rest in the Profile. Latest version of FsLogix.

    Very spooky scenario..!

    Tuesday, August 11, 2020 10:14 PM
  • Good morning,

    When you say per machine  or per user what do you mean ?

    Because in our RDS server I can only see that option to install OneDriveSetap.exe /allusers

    What did tou specify as arguments for the installation of OneDrive.

    Do your users make use of shared folders created in Microsoft Teams and then synchronize in OneDrive  with files on  demand enabled ?

    Is with this specific deployment we detected the issues.

    We did not have the time to verify if OneDrive user folder of were affected of this strange behaviour.

    Monday, August 17, 2020 8:44 AM
  • Hi,

    We do not run the OneDriveSetap.exe /allusers at all.

    We are just using a simple powershell logon script that checks if the OneDrive already is installed in the users profile.

    C:\Users\$user\AppData\Local\Microsoft\OneDrive\OneDrive.exe

    if not installed, the installation runs with the silent switch in users context.

    \\sharedPath\OneDriveSetup.exe /silent

    I`m not sure about the "folders created in Microsoft Teams", but users are using the FilesOnDemand all the time.

    Wednesday, August 19, 2020 11:18 AM
  • Just wanted to report that once we changed the GPO to not include OneDrive in Office 365 container, we no longer have the issue with corruption / file name changes / file moves / deleted files.  OneDrive outside of the 365 Container runs as expected.
    Monday, August 24, 2020 4:19 PM
  • So where is this data now stored? Inside the Profile Container? 
    Saturday, August 29, 2020 11:37 AM
  • Hi Southland_Admin,

    how dit you deploy OneDrive in your TS environment? With /alluseres or a per user basis.

    @Frederik_E: do you deploy OneDrive with "change user /install" and "change user /execute"? or do you execute the silent installation without using the install method for TS enviroments?

    Thank you for your time and kind regards.

     

    Saturday, September 5, 2020 4:34 PM
  • Any update on this? We had a number of customer running stable for months.. now implmented a new customer on the same platform, that keeps having this issue. 
    Monday, September 14, 2020 5:53 AM
  • I am also getting this issue...opened a MSFT case. Nothing back yet....does anyone have a fix? 
    Thursday, September 17, 2020 3:32 PM
  • Yes, its placed in the Profile Container rather than the Office 365 Container and since making that change, we no longer have the issue.  To my understanding, there's no downside of placing it in the Profile Container.  In fact, it's really up to you if you've rather have all components reside in the Profile Container or split the respective components into different containers.
    Thursday, September 24, 2020 8:53 PM
  • Per machine.  OneDriveSetup.exe /allusers
    Thursday, September 24, 2020 8:54 PM
  • I have received a hotfix release for my case. I am validating if the problems are solved.
    Version FSLogix_Apps_2.9.7486.53382
    Wednesday, September 30, 2020 1:27 PM
  • We are facing the same issue with Files On Demand with the latest version of FSlogix 2.9.7349.30108

    We are getting black screens after the first login that presents for over 10 minutes. Then OneDrive can no longer sync. We have tried it with OneDrive in the Office Container and also in the Profile Container.

    Did FSLogix_Apps_2.9.7486.53382 resolve your issues with OneDrive and syncing with 365 SharePoint sites?

    Thanks


    • Edited by JesseBoehm Thursday, October 1, 2020 9:01 PM
    Thursday, October 1, 2020 3:23 PM
  • Hi! Where did you get\download this version - FSLogix_Apps_2.9.7486.53382?
    Sunday, October 4, 2020 5:28 PM
  • @Southland_Admin: Yes, just like this: \\sharedPath\OneDriveSetup.exe /silent

    No use of the "Change user". This is in user context.

    • Edited by Fredrik_E Sunday, October 4, 2020 8:46 PM
    Sunday, October 4, 2020 8:44 PM
  • FSLogix_Apps_2.9.7486.53382 wont' solve this problem


    Monday, October 26, 2020 8:37 AM
  • Hi, have you tested this latest version?

    Release notes for FSLogix Apps release PUBLIC PREVIEW 2009 (2.9.7576.47294)


    Have you also tested saving OneDrive to Profile Containers?

    Monday, October 26, 2020 8:40 AM
  • We've been having the same issue.... I first noticed when we were deploying an RDS pool in Azure, but now we're facing the same issue on WVD. 

    I've been able to replicate the issue and it seems to be related (in our case) if we're using Profile and Office containers and within the FsLogix Office 365 Container configure 'Include OneDrive data in container' - Enabled and 'Include SharePoint data in container' - Enabled. If we disable one, the random corruption/moving of files and the 10 minutes black screen does not occur. 

    Setup is OneDrive latest version installed with /allusers, latest (2.9.7349.30108) version FsLogix. I'm curious if anybody else who's facing this issue have the same GPO settings.

    -- EDIT --

    I've been doing some more testing and it looks like SharePoint data is not redirected to the Office Container, but is stored in the Profile ontainer. I did specify in the Offce GPO that SP data is Enabled.

    • Edited by Alex Gnocchi - KNS Wednesday, October 28, 2020 12:02 PM Result from extra testing
    Wednesday, October 28, 2020 8:52 AM
  • I have had exactly this same issue with three seperate FSLogix deployments this month.

    2 x Windows server 2019 deployments and 1 X WVD deployment in Azure.

    I have tried turning off or on files on demand and this doesn't seem to make a difference.

    I have also tried configured FSLogix to store onedrive (and sharepoint) data in either the profile container or office container and this has also not made a difference.

    Does anyone have any suggestions as to what i can do about this as it is seriously making FSLogix unusable for any customers using Sharepoint.

    Interestingly i have also noticed when this has happened that it seems to mix a users private onedrive data into the sharepoint sites that are being synced which is even more concerning :(

    I have attempted to get some support for the issue from microsoft but was literally told that it was unsupported and that cases could not be raised to the FSLogix team.

    Very disappointing!

    Friday, November 20, 2020 9:14 AM