Topic
  • 8 replies
  • Latest Post - ‏2014-01-09T18:30:33Z by TKAJ_Suresh_Punuru
CurtisMiles
CurtisMiles
3 Posts

Pinned topic Republish Hangs after lookup of Derby DataSource

‏2013-07-01T16:28:39Z |

Hello!

I've been to use the WAS 8.5.5. Liberty Profile for some dev work I've been doing.  Things have been great except for the painful fact that republishing after application updates has been hanging, forcing me to kill the server and restart again every time I make changes.  My application is a simple web app with a servlet that pulls a few records from a Derby DB, running in Network Server mode (version 10.10.1.1).

After some troubleshooting, I've identified that the problem only happens after the following line executes in my web app:

     myDataSource = (DataSource)new InitialContext().lookup("jdbc/MyDatabase");

If I invoke my servlet from a web browser with the above line commented out, I can make changes to my app and republish just fine. But as soon as that line executes, all future republishing fails.  Note that my application works fine after restarting the server (i.e. the datasource can be successfully found and used), but republishing fails thereafter.

My server.xml defines my datasource like this:

    <variable name="DERBY_JDBC_DRIVER_PATH" value="D:/WAS-Liberty-Profile_8.5.5/db-derby-10.10.1.1-bin/lib"/>
    <library id="derbyLib">
       <fileset dir="${DERBY_JDBC_DRIVER_PATH}"/>
    </library>
    <dataSource id="MyDerbyDatasource" jndiName="jdbc/MyDatabase" statementCacheSize="10">
       <jdbcDriver libraryRef="derbyLib"/>
       <properties.derby.client createDatabase="false" databaseName="D:/WAS-Liberty-Profile_8.5.5/db/MyDB" serverName="localhost" user="APP" password="{xor}Hg8P"/>
    </dataSource>
 

After it hangs, I see the following message in the server console, but that's where it sits...:

[AUDIT   ] CWWKT0017I: Web application removed (default_host): http://MyHostName:9080/MyApp/

 

Can anyone help me?  Having to continually kill my server is killing *me*!  :)

Thanks,
Curtis.

 

  • frowe
    frowe
    7 Posts

    Re: Republish Hangs after lookup of Derby DataSource

    ‏2013-07-03T11:16:59Z  

    Curtis,

    When the republish hangs, have you been making code changes to your application or making configuration changes to server.xml ?

    Fred

  • CurtisMiles
    CurtisMiles
    3 Posts

    Re: Republish Hangs after lookup of Derby DataSource

    ‏2013-07-03T14:22:36Z  
    • frowe
    • ‏2013-07-03T11:16:59Z

    Curtis,

    When the republish hangs, have you been making code changes to your application or making configuration changes to server.xml ?

    Fred

    Hi Fred,

    Thanks for your response.  No... No changes to server.xml.  I'm just changing my servlet code, although the actual code change doesn't seem to matter.  As soon as I hit my servlet and cause the code to execute the looks up the data source, any future changes that require republishing cause the server to hang.

    As an additional clue, after the republishing hangs I can no longer access my application through a browser.  So it looks like the application is successfully removed from the server (although perhaps not completely?)

    Thanks,
    Curtis.

  • djans
    djans
    1 Post

    Re: Republish Hangs after lookup of Derby DataSource

    ‏2013-09-12T14:32:00Z  

    Did you get this resolve as i am having the same problem but with JMS connection ( to MQ )... The common factor would be the JNDI lookup ?

     

    Thanks.

  • CurtisMiles
    CurtisMiles
    3 Posts

    Re: Republish Hangs after lookup of Derby DataSource

    ‏2013-09-13T15:14:53Z  
    • djans
    • ‏2013-09-12T14:32:00Z

    Did you get this resolve as i am having the same problem but with JMS connection ( to MQ )... The common factor would be the JNDI lookup ?

     

    Thanks.

    Unfortunately I never got this resolved... Still seeing the issue whenever the JNDI lookup is done on the data source... :(

  • TomBanks
    TomBanks
    13 Posts

    Re: Republish Hangs after lookup of Derby DataSource

    ‏2013-09-16T15:26:29Z  

    Unfortunately I never got this resolved... Still seeing the issue whenever the JNDI lookup is done on the data source... :(

    Hi Curtis,

    I'll see if there is someone in the dev team familiar with JDBC to have a look at this.

    Thanks,

    Tom

  • njr11
    njr11
    14 Posts

    Re: Republish Hangs after lookup of Derby DataSource

    ‏2013-09-16T15:52:02Z  
    • TomBanks
    • ‏2013-09-16T15:26:29Z

    Hi Curtis,

    I'll see if there is someone in the dev team familiar with JDBC to have a look at this.

    Thanks,

    Tom

    Would it be possible for either of you to capture a server dump (which will include information about what threads are doing) during the time that it is hung, and attach the output to a reply?

    The following command can be used,

    wlp/bin> server dump name_of_your_server

    http://pic.dhe.ibm.com/infocenter/wasinfo/v8r5/index.jsp?topic=%2Fcom.ibm.websphere.wlp.nd.doc%2Fae%2Ftwlp_setup_dump_server.html

    If you don't feel comfortable attaching the entire server dump, I'll also point out the file within it which has the thread information and is most likely to be useful in debugging this is ThreadInfoIntrospector.txt

  • M9HU_Stuart_Charlton
    1 Post

    Re: Republish Hangs after lookup of Derby DataSource

    ‏2013-09-21T23:18:49Z  
    • njr11
    • ‏2013-09-16T15:52:02Z

    Would it be possible for either of you to capture a server dump (which will include information about what threads are doing) during the time that it is hung, and attach the output to a reply?

    The following command can be used,

    wlp/bin> server dump name_of_your_server

    http://pic.dhe.ibm.com/infocenter/wasinfo/v8r5/index.jsp?topic=%2Fcom.ibm.websphere.wlp.nd.doc%2Fae%2Ftwlp_setup_dump_server.html

    If you don't feel comfortable attaching the entire server dump, I'll also point out the file within it which has the thread information and is most likely to be useful in debugging this is ThreadInfoIntrospector.txt

    I have the same problem as the OP, but in my case, I'm using Derby embedded driver - same version 10.10.1.1.

    In my case, immediately after starting the server after changing a WAR file, the application is removed, and the server stops shuts down immediately.    This even occurs across server stop/starts when the application monitor updateTrigger is disabled.   

    One quirk is that this never happens if I start or stop the server from the command shell, updating the WAR manually.  It only happens when I'm running the start/stop from a deployment script.

    The workaround is to start the server again manually.

     

    server.xml:

    <applicationMonitor updateTrigger="disabled"/>

    <dataSource jndiName="mobile/jdbc/WorklightDS" transactional="false"

                                    statementCacheSize="10">

                    <jdbcDriver libraryRef="derby"

                    javax.sql.ConnectionPoolDatasource="org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource"     />

     

                    <properties.derby.embedded databaseName="/opt/mobile/derby/worklight/worklight" user="worklight" shutdownDatabase="false"

                            connectionAttributes="upgrade=true"/>

                    <connectionManager connectionTimeout="180"

                            maxPoolSize="10" minPoolSize="1"

                            reapTime="180" maxIdleTime="1800"

                            agedTimeout="7200" purgePolicy="EntirePool" />

    </dataSource>

    logs:

    [9/21/13 18:55:01:432 EDT] 00000016 com.ibm.ws.app.manager.internal.statemachine

    .StartAction     A CWWKZ0001I: Application TestApp started in 15.125 se

    conds.

    [9/21/13 18:55:01:453 EDT] 0000001a com.ibm.ws.kernel.feature.internal.FeatureMa

    nager            I CWWKF0008I: Feature update completed in 17.123 seconds.

    [9/21/13 18:55:01:454 EDT] 0000001a com.ibm.ws.kernel.feature.internal.FeatureMa

    nager            A CWWKF0011I: The server TestApp is ready to run a sma

    rter planet.

    [9/21/13 18:55:01:566 EDT] 00000034 com.ibm.ws.http.internal.VirtualHostImpl

                     A CWWKT0017I: Web application removed (default_host): http://10

    .195.1.18:6080/mobile/

    [9/21/13 18:55:01:669 EDT] 00000034 com.ibm.ws.app.manager.internal.statemachine

    .StopAction      A CWWKZ0009I: The application TestApp has stopped succ

    essfully.

    [9/21/13 18:55:01:729 EDT] 00000035 com.ibm.ws.tcpchannel.internal.TCPChannel

                     I CWWKO0220I: TCP Channel defaultHttpEndpoint has stopped liste

    ning for requests on host *  (IPv6) port 6080.

    [9/21/13 18:55:01:733 EDT] 00000035 com.ibm.ws.tcpchannel.internal.TCPChannel

                     I CWWKO0220I: TCP Channel defaultHttpEndpoint-ssl has stopped l

    istening for requests on host *  (IPv6) port 6443.

    [9/21/13 18:55:01:812 EDT] 00000034 com.ibm.tx.jta.impl.FailureScopeController

                     A WTRN0105I: The transaction service has shutdown successfully

    with no transactions requiring recovery.

    [9/21/13 18:55:02:019 EDT] 00000001 com.ibm.ws.kernel.launch.internal.FrameworkM

    anager           A CWWKE0036I: The server TestApp stopped after 19.609

    seconds.

     

  • TKAJ_Suresh_Punuru
    TKAJ_Suresh_Punuru
    1 Post

    Re: Republish Hangs after lookup of Derby DataSource

    ‏2014-01-09T18:30:33Z  
    • djans
    • ‏2013-09-12T14:32:00Z

    Did you get this resolve as i am having the same problem but with JMS connection ( to MQ )... The common factor would be the JNDI lookup ?

     

    Thanks.

    Thanks a lot. It saved a lot of time for me. As you pointed out, using JNDI caused the republish issue on Liberty profile server. I had the same problem as I was using Oracle database configured as JNDI data source.

    I commented out jndi data source and configured spring data source implementation for development purpose as shown below in root context.xml. I am able to republish code now. When build the project to deploy on test servers, I can switch back to JNDI lookup.

    <!--  
        <jee:jndi-lookup id="TestDB" jndi-name="jdbc/TestDB" />
    -->
        
        <bean id="TestDB" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
            <property name="driverClassName" value="oracle.jdbc.OracleDriver"/>
            <property name="url" value="jdbc:oracle:thin:@blah.blah"/>
            <property name="username" value="user"/>
            <property name="password" value="password"/>
        </bean>