Retrieve logs faster RRS feed

  • Question

  • From: V Murali K Raju @VMuraliKRaju via Twitter

    How to retrieve the hadoopservicelog faster way, want to retrieve three months log, it is taking more than 12 hours but not extracted completely so far? retrieve of logs through visual studio option taking huge amount time, customer will not going to wait for analysis. can i filter on basis date time? If so, please let me know syntax with example.



    Friday, June 3, 2016 7:40 PM

All replies

  • Hi Dani,

    Thank you for posting here!

    hadoopservicelog: Log of events/exceptions recorded by all Hadoop services. This table may be useful in debugging issues related to job failures on HDInsight clusters.

    There are many tools available for accessing data in these tables:

    • Visual Studio
    • Azure Storage Explorer
    • Power Query for Excel

    You can now use Excel to filter and sort as necessary. Obviously, you may want to include other columns (e.g. Message) in order to drill down into issues when they occur, but selecting and grouping the columns described above provides a decent picture of what is happening with Hadoop services.

    For more information, kindly refer “Analyze HDInsight logs”.





    If a post answers your question, please click Mark as Answer on that post and Vote as Helpful.

    Saturday, June 4, 2016 4:33 AM
  • Your suggested solution have tried using Visual Studio. to retrieve 2 months of data it took 12 hours. I need specific day data. Have tried to use timestamp to filter but filter not working.

    One more question - Have installed power query patch to use excel but not getting azure datasource in excel. Can you suggest me solution for this also

    Azure storage explore



    Saturday, June 4, 2016 9:00 AM
  • Hi Murali,

    In Visual Studio you can use Constructing filters.

    To filter data in an Azure table that is displayed in the Visual Studio Table Designer, you construct a filter string and enter it into the filter field. The filter string syntax is defined by the WCF Data Services and is similar to a SQL WHERE clause, but is sent to the Table service via an HTTP request. The Table Designer handles the proper encoding for you, so to filter on a desired property value, you need only enter the property name, comparison operator, criteria value, and optionally, Boolean operator in the filter field.

    For more information about constructing filters, see Construct Filter Strings for the Table Designer.


    This document gives the step by step procedure for how to use power query to open and analyze the service log.

    Here is a document for “Azure Storage explore”.




    Saturday, June 4, 2016 9:53 AM