none
Memory leak in MS TFS Java SDK 1.0

    Soru

  • Hi,

    We have a stateless application that operates as  a Windows service. A task scheduler schedules the application to run once every 15 minutes.

    The application -

    • Opens a new TFSProjectCollection.
    • Queries/Updates TFS with data.
    • Closes/shuts down the Collection.

    We observed rapid memory growth of around 300M/hour, with 3 major leak suspects:

    1. org.hsqldb.Memorynode
    2. ShutdownManager.
    3. org.apache.log4j.consoleappender,

    all shipped as part of the TFS API. 

    We could resolve issues 1 and 2 and need suggestions on 3. Please also confirm the changes for 1 and 2 , and suggest better alternatives, if any.

     

    Step 1: Org.hsqldb.Database leaking, reconfigure db.properties:

    Upon decompiling the TFS API completely, and researching hypersonic SQL further, I could avoid the memory leak with the hsqldb library. The workaround is to set a system property explicitly in the jar, by setting all CREATE TABLE types to "cached" instead of "memory"(the former option leaks hsqldb). 

    This could be done by modifying their jdbc connection url jdbc.connection.url=jdbc:hsqldb:LOCAL_SETTINGS_DISKFILE;hsqldb.default_table_type=cached, in Microsoft-tfs-sdk-1.0.jar/db.properties.

    Unfortunately, this fix merely mitigated the problem as the application shutdown hook of the API still leaked, there are multiple shutdown manager objects that still hog memory. I had to apply Step2 to resolve the problem further.

     

    Step 2: ShutDownManager.getInstance().shutdown():

    I figured out a teardown API that fires all shutdown() listeners and releases memory held by shutdown() objects. The plugin is not leaking after adding calls to the teardown API at strategic points in the program flow(‘finally’ clauses). The instance is running fine so far.

     

    Step 3:log4j:

     

    After the memory leak was mitigated by a very great extent(about 95%), I see that the TFS API still latches on to references of org.apache.log4j.consoleappender. I believe this can be worked around by setting the correct log4j properties(the Microsoft API uses log4j for logging). We’d need Microsoft’s suggestions on that. The log4j.properties within the jar needs to be reconfigured.

     

    To summarize, we need Microsoft to validate the workarounds in Steps 1/2/3 and suggest the optimal settings for log4j.properties as part of step 3.


    • Düzenleyen PadmarajKB 29 Aralık 2011 Perşembe 06:37 More info on memory leak
    • Taşıyan Vicky SongMicrosoft employee 30 Aralık 2011 Cuma 06:57 TFS integrated with Java (From:Team Foundation Server - General)
    29 Aralık 2011 Perşembe 06:32

Yanıtlar

  • Your reconfiguration of HSQLDB looks OK.  HSQLDB defaults to MEMORY tables to increase performance, but choosing CACHED should be completely compatible with the TFS SDK for Java and TEE clients.  The SDK finds the "db.properties" file using the com.microsoft.tfs.core.internal.db.ConnectionConfiguration class's classloader, so you can put your own version on the classpath instead of modifying the product JAR.  We haven't researched the performance tradeoffs of MEMORY versus CACHED table types.

    I'm curious what kind of objects were left over in the ShutdownManager.  TFSConnection.close() should remove all the shutdown listeners for that connection and that connection's clients (TFSConnection's active object graph is quite large).  TempStorageBroker, being a singleton, keeps its listener attached forever, but you can safely clear out the on-disk temp files with a call to TempStorageBroker.cleanUpAllItems().

    I don't quite understand what you're trying to accomplish with the logging system.  Just release all the memory it was using?  The SDK classes use Apache Commons Logging as the interface for all logging, and the implementation we ship and use by default is log4j.  Our classes use an ACL LogFactory to get an ACL Log object that's kept in a static field, which is never closed or freed.  Commons Logging's LogFactory class seems to keep references to the Logs it creates (lots of them for all the classes in the SDK).  Have you tried calling LogFactory.release()?  That might free a lot of references.

    The SDK includes some sample code on configuring log4j in the SDK.  Take a look at LogConfigurationSample.java in the com.microsoft.tfs.sdk.samples.console sample project.  The Javadoc for the com.microsoft.tfs.logging.config.Config class goes has more details on static and dynamic configuration options.
    05 Ocak 2012 Perşembe 14:04
    Sahip

Tüm Yanıtlar

  • Hello PadmarajKB,

    I am moving your case to the Team Foundation Server - Eclipse and Cross Platform forum so that you can get better support.

    Thanks.


    Vicky Song [MSFT]
    MSDN Community Support | Feedback to us
    30 Aralık 2011 Cuma 06:56
  • Your reconfiguration of HSQLDB looks OK.  HSQLDB defaults to MEMORY tables to increase performance, but choosing CACHED should be completely compatible with the TFS SDK for Java and TEE clients.  The SDK finds the "db.properties" file using the com.microsoft.tfs.core.internal.db.ConnectionConfiguration class's classloader, so you can put your own version on the classpath instead of modifying the product JAR.  We haven't researched the performance tradeoffs of MEMORY versus CACHED table types.

    I'm curious what kind of objects were left over in the ShutdownManager.  TFSConnection.close() should remove all the shutdown listeners for that connection and that connection's clients (TFSConnection's active object graph is quite large).  TempStorageBroker, being a singleton, keeps its listener attached forever, but you can safely clear out the on-disk temp files with a call to TempStorageBroker.cleanUpAllItems().

    I don't quite understand what you're trying to accomplish with the logging system.  Just release all the memory it was using?  The SDK classes use Apache Commons Logging as the interface for all logging, and the implementation we ship and use by default is log4j.  Our classes use an ACL LogFactory to get an ACL Log object that's kept in a static field, which is never closed or freed.  Commons Logging's LogFactory class seems to keep references to the Logs it creates (lots of them for all the classes in the SDK).  Have you tried calling LogFactory.release()?  That might free a lot of references.

    The SDK includes some sample code on configuring log4j in the SDK.  Take a look at LogConfigurationSample.java in the com.microsoft.tfs.sdk.samples.console sample project.  The Javadoc for the com.microsoft.tfs.logging.config.Config class goes has more details on static and dynamic configuration options.
    05 Ocak 2012 Perşembe 14:04
    Sahip