locked
Is Microsoft Really Getting A Representative Sample from All of this Telemetry Data?

    General discussion

  • It seems clear from the Building Windows 8 entries that a great deal of the design of Windows 8 is coming from telemetry data collected from Windows 7 users.  However, it seems to me that there is no way this would be a good representative sample.

     

    If you're trying to see if someone is using an advanced feature, and get a response that nobody is, is that really accurate?  I mean, every basic computer user is going to leave the data collection enabled since they don't even know it's there, or how to turn it off.  Of course these people aren't using an advanced feature.  Advanced users, on the other hand, are much more likely to turn off these data collection options, which means that the process itself is filtering out the very users who would be helpful to have the opinion of.

     

    I'd love to hear if anyone agrees or disagrees.

    Wednesday, October 19, 2011 6:21 AM

All replies

  • Yes, I think you've hit upon a genuine flaw in the data telemetry data being used.  And no, you are not the only one to notice that the data is skewed. 

    Wednesday, October 19, 2011 2:47 PM
  • I agree.   I'm an experienced user and disable all that kind of tracking stuff whereas an inexperienced user will leave it enabled.

    One thing that has always bothered me about Windows is that there is no way to click 'I am an experienced user' checkbox somewhere to disable all the dumbed-down settings which are enabled by default.

     

     

    Wednesday, October 19, 2011 3:34 PM
  • Sinofsky just claimed that the telemetry data is "a full census without any inherent bias."  It's in the comments section:

    http://blogs.msdn.com/b/b8/archive/2011/10/18/designing-search-for-the-start-screen.aspx

     

    Wednesday, October 19, 2011 7:34 PM
  • As an advanced user and sysadmin I've always kepy CEIP data switched on, because I know it helps the future of Windows development and has no realistic privacy implications. You may well be right that overly paranoid users aren't going to have contributed to the telemetry data, but that does not translate to only data from inexperienced users.
    Wednesday, October 19, 2011 8:23 PM
  • dunfield:

    I'm pretty well certain the option doesn't exist because of a nasty thing known as the human ego. 

    There are people that will consider themselves a triumphant powerhouse of IT knowledge if they've learned to use keyboard shortcuts.

    Others will make this leap to "computing guru" once they discover the task manager or event viewer.

    More still will consider themselves expert if they are able to Bing or Google search for issues they are having and follow neatly written step-by-step instructions on how to solve common problems.

    Consider the following:

    You have a room filled with 100 people, 25 groups of 4 each based on their number of years of experience with computers.

    If you ask them to describe their level of proficiency, my bet is you'll get at least 95% describing themselves as "expert". 

    If you have these same 100 people sit down and memorize the pool of answers for a particular MCITP exam and they pass, you have a bunch of people who will absolutely, loudly and triumphantly in fact, declare themselves to be shining stars of the IT sub-culture. 

    Not accusing anyone here, but I see examples of the last point each and every day.  I've come to discover the more people talk about "certs", and the more of those "certs" they have neaty framed and edge-aligned on their cubicle walls, the more likely you are to be interacting with what amounts to a neophyte.

    A neophyte with "papers" to prove he "knows his stuff".

    DAS

    Thursday, October 20, 2011 11:41 AM
  • @Win7Tester 

     

    There ya go ... according to your numbers 95% would probably want the 'I'm an expert' checkbox ... whether they should check it or not.  If they get lost, they can always UNcheck it again.  At least the rest of us won't feel like we're being treated like children our entire computing lives.  ;)

    I don't know about their user data.  There seem to be an awful lot of people who are reacting negatively to the changes in the start menu which, according to their numbers, is supposed to benefit most people.  It's cute; it's novel but more productive?  I don't think so.  I'm one of those who have *no* shortcuts on his desktop so forcing me to interact with an interface that looks like a cluttered game of colour dominoes might be enough to make me revert back to XP.  ;)

    Thursday, October 20, 2011 1:28 PM
  • Please don't give Apple a bad name by comparing their work to Windows 8. Windows 8 is not designed for desktop and if Microsoft comes out and says it is with straight face then more power to the liars.

    I would compare few aspects of windows 8 with OSX Lion which has some of similar changes but how Apple again gets it right and Microsoft does not. This is from desktop/notebook perspective and not tablet.

    Start Menu, Dock and LaunchPad

    - OSX does not alienate its existing user as it retains the Doct and all the benefit it has but adds a LaunchPad to launch apps which only appears upon certain key stroke.

    - The dock still remains visible when LaunchPad is displayed.

    - The desktop is not an app but it is the computer.

    Full Screen Applications

    - OSX did the full screen apps designed right. It allows same application to run both in desktop and full screen mode. This provides all the benefits of full screen experience without jeopardising the non-full screen experience.

    - Again Windows messed up here by having full screen apps on desktop work same as for tablet with charms bar and other similar tablet constructs .

    So the two main features on desktop are much better on OSX. This tells me why Apple is a leader in usability while Microsoft lags behind.

    Here is what Microsoft should have done for Windows:

    Applications:

    - Each applications should be able to run in 3 (or fewer) modes: Desktop, Full Screen Desktop and Tablet.

    - In desktop mode, it should keep same consistent user interface as other desktop applications.

    - In full screen desktop mode, it should provide immersive experience with menu available on some key press or something.

    - For tablet device, the app should provide full screen experience with charms bar and all other finger friendly goodies ;)

    Start Menu:

    - Keep windows start menu

    - Use Aero peek for tiles (similar to gadgets)

    - Allow users to put gadgets with tiles on same screen or divide the peek button in two parts, one for desktop and one for tiles (or launchpad)

    - Allow searching/launching of all applications (both regular and metro) via traditional start menu or new tiles.

    - Always keep the task bar visible, in each screen until user goes full screen application mode.

    For tablet:

    The start menu can default to tiles screen and use that as the only experience and don't show the taskbar if user is running in tablet mode.

    This would improve and enhance desktop experience and provide clean tablet experience. It would provide choice to users who don't want to use tiles on desktop. This would make the OS have multiple personalities each designed for specific experience where underlying foundations being the same.

    It would also make it easy to develop one application for experinece across multiple set of devices.

    Please do something like this, please don't make us hate desktop

    Thursday, October 20, 2011 2:15 PM
  • I'm gonna try and say this as respectfully as possible...

    Keep your Apple garbage out of the msdn forums please.

    We are discussing microsoft products here and most of us don't care about what apple is doing.

    Microsoft should focus on building a great windows product and it's pretty much irrelavent what apple is doing as they are (contrary to popular opinion) in completely different markets.

    Microsoft makes software and an operating system to run on and work with computers.

    Apple makes closed devices, much like Sony makes a playstation.

    Different markets, different target audiences, different strategies. IF msft looks at what apple is doing, I'ld personally hope that the underlying goal would be to learn how NOT to do it.

    Screw apple an its dictatorship, this is microsoft baby!

    Thursday, October 20, 2011 2:35 PM
  • @dunfield

    "might be enough to make me revert back to XP.  ;)"

    You just lost all creditably. The fact that anyone would consider XP as an option is just silly.
    Thanks,
    Bobby Cannon
    BobbyCannon.com
    Friday, October 21, 2011 2:20 AM
  • It seems clear from the Building Windows 8 entries that a great deal of the design of Windows 8 is coming from telemetry data collected from Windows 7 users.  However, it seems to me that there is no way this would be a good representative sample.

     

    If you're trying to see if someone is using an advanced feature, and get a response that nobody is, is that really accurate?  I mean, every basic computer user is going to leave the data collection enabled since they don't even know it's there, or how to turn it off.  Of course these people aren't using an advanced feature.  Advanced users, on the other hand, are much more likely to turn off these data collection options, which means that the process itself is filtering out the very users who would be helpful to have the opinion of.

     

    I'd love to hear if anyone agrees or disagrees.

    We turn off CEIP as much as possible on our 10,000 computers with Windows and Office.  (I think you're referring to this?)  No, we don't like being tracked, but mainly we do this because there's no need for useless data being transmitted from thousands of computers across our network that is unnecessary for our purpose.  If Microsoft is really interested in how people use computers in a corporate environment like ours, then they should ask instead of collecting anonymous data from a computer.

    How many pixels we move our mouse to get to the Start Menu, or how often we open the Start Menu is only part of the overall experience, or functionality we need available for using a computer.  But it seems this is the only part they are basing their Windows 8 interface from, so I have to agree with you that their approach is flawed and the end result is a disfunctional experience with the interface in Windows 8 for desktop/laptop users.

    Friday, October 21, 2011 1:48 PM
  • @dunfield

    "might be enough to make me revert back to XP.  ;)"

    You just lost all creditably. The fact that anyone would consider XP as an option is just silly.
    Thanks,
    Bobby Cannon
    BobbyCannon.com
    You may have missed the ironic smiley at the end.  ;)
    Friday, October 21, 2011 2:12 PM
  • So ergo: don't disable sending telemetry data.

    Why would somebody need to disable it unless your are using illegal version? And even if you do (I am in either way saying that YOU use something illegaly; so please do understand me correctly here), I don't think MSFT would care about unwinding any traces up until it finds you. I bet MSFT has something better to be bothered with.

    So please don't disable telemetry. It does not harm your privacy. I bet, if you install a good packet/connection analysis software you'd find lots of 3rd party apps that actually do send even some memory dumps, which hold much more about your privacy than that 'what controls used the user' telemetry the OS is sending back to MSFT.

     


    Well this is the world we live in And these are the hands we're given...
    Saturday, October 22, 2011 7:43 AM
  • I'm gonna try and say this as respectfully as possible...

    Keep your Apple garbage out of the msdn forums please.

    We are discussing microsoft products here and most of us don't care about what apple is doing.

    Microsoft should focus on building a great windows product and it's pretty much irrelavent what apple is doing as they are (contrary to popular opinion) in completely different markets.

    Microsoft makes software and an operating system to run on and work with computers.

    Apple makes closed devices, much like Sony makes a playstation.

    Different markets, different target audiences, different strategies. IF msft looks at what apple is doing, I'ld personally hope that the underlying goal would be to learn how NOT to do it.

    Screw apple an its dictatorship, this is microsoft baby!

    <Remove Personal Comments>

     

    My post was to contrast competing features in OSX and Windows 8. Apple did not dumb down the desktop where as Microsoft is doing that. Given that desktop users are probably 20x more than tablet, this seems like a bad design choices from Microsoft.



    Saturday, October 22, 2011 10:28 AM
  • I would agree that there is a flaw in the over zealous use of the telemetry data.  Microsoft has stated in their blogs that they address Corporations by discussions with Corporate accounts.  I would have to state that even if this were done that the data from an IT CEO will not effectively represent the real issues encountered in day to day use of the computer.  It would be highly influenced by personal use or the few squeaky wheels within the Corporation.  The other issue is how many actual machines are NOT sending the data because the process on the PC has failed.  I have found events stating that data could not be sent and Microsoft's overall response has been that these errors are not critical to the operation of the machine therefore the best approach is to turn off the feature. 
    Sunday, October 23, 2011 10:00 PM