[go: up one dir, main page]
Menu ▾ ▴

#303 restart wrapper doesn't work on ubuntu 16.04 LTS

v3.5.13
open
nobody
None
5
2022-05-27
2017-08-23
Bin Yi
No

After WrapperManager.restart() call, the JVM is stopped but never start again.
Tested exactly same code/config working fine on ubuntu 14.04 but not 16.04.
By enable the debug log of wrapper, the log stop after these lines:

INFO | jvm 1 | 2017/08/23 00:31:31 | WrapperManager Debug: calling System.exit(0)
INFO | jvm 1 | 2017/08/23 00:31:31 | WrapperManager Debug: Server daemon shut down
DEBUG | wrapper | 2017/08/23 00:31:32 | Signal trapped. Details:
DEBUG | wrapper | 2017/08/23 00:31:32 | signal number=17 (SIGCHLD), source="unknown"
DEBUG | wrapper | 2017/08/23 00:31:32 | Received SIGCHLD, checking JVM process status.
DEBUG | wrapper | 2017/08/23 00:31:32 | JVM process exited with a code of 0, leaving the wrapper exit code set to 0.
DEBUG | wrapper | 2017/08/23 00:31:32 | JVM exited normally.
DEBUG | wrapper | 2017/08/23 00:31:32 | Preparing to restart with mode 4.
DEBUG | wrapper | 2017/08/23 00:31:32 | Waiting 3 seconds before launching another JVM.

Discussion

  • Leif Mortenson

    Leif Mortenson - 2017-08-23

    The final line in the log is "Waiting 3 seconds before launching another JVM.".
    What happens in the log next? This log shows me that the Wrapper is about to restart as expected.

    Was that the end of the log file? If so, was the Wrapper process still running?

    Cheers,
    Leif

     

  • Bin Yi

    Bin Yi - 2017-08-23

    that's the last line of the log. and after that the wrapper process is gone:

    ubuntu@ip-172-31-8-191:/usr/local$ ps -A | grep wrapper <--- before restart
    4092 ? 00:00:30 wrapper
    ubuntu@ip-172-31-8-191:/usr/local$ ps -A | grep wrapper <-- after restart
    ubuntu@ip-172-31-8-191:/usr/local$

    Thanks,
    Bin

     

  • Maxime Andrighetto

    Maxime Andrighetto - 2017-08-25

    Thank you for your reply.
    Are you running the Wrapper as a daemon or in the terminal?
    If possible, please send us the configuration file and the log file to support@tanukisoftware.com so that we can check what is wrong.
    Please note that 3.5.13 is quite old (it was released in 2011). Could you try with the latest version of the Wrapper (3.5.33) and see if the problem persists?

    Best Regards,

    Maxime,
    Java Service Wrapper Team

     

  • Gourav Garg

    Gourav Garg - 2021-03-08

    @maxime-tsl @mortenson
    I am getting this on 3.5.44 and ubuntu 18 os
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Attempt to load native library with name: libwrapper-linux-x86-64.so Result: Success!
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Successfully loaded native library.
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Loaded localized resources.
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Calling native initialization method.
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperJNI Debug: Inside native WrapperManager initialization method
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Java PID : 3781
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Java Version : 1.8.0_275-b01 OpenJDK 64-Bit Server VM
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Java VM Vendor : Amazon.com Inc.
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: OS Name : Linux
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: OS Arch : amd64
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug:
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Control event monitor thread started.
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Startup runner thread started.
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: WrapperManager.start(a org.tanukisoftware.wrapper.WrapperSimpleApp, args[]) called by thread: main
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Initial thread: main Priority: 5
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Communications runner thread started.
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Open socket to Wrapper...Wrapper-Connection
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Unable to open socket to Wrapper from port 31,000, already in use.
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Opened Socket from 31,001 to 32,000
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Send a packet KEY : nJom2_Y_wlKb7TTWd4cEWyXsTFo8zqXL
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager: ERROR - The version of the Wrapper which launched this JVM is "3.5.33"
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager: while the version of the Wrapper jar file currently in use
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager: is "3.5.45".
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager:
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager: ERROR - The version of the Wrapper which launched this JVM is
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager: "3.5.33-st" while the version of the native library
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager: is "3.5.45-st".
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager:
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: WrapperManager.stop(1) called by thread: Wrapper-Connection
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Send a packet STOP : 1
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Pausing for 1,000ms to allow a clean shutdown...
    DEBUG | wrapperp | 2021/03/08 11:25:33 | accepted a socket on port 32000 from 127.0.0.1 at port 31001
    DEBUG | wrapperp | 2021/03/08 11:25:33 | closing backend server.
    DEBUG | wrapperp | 2021/03/08 11:25:33 | read a packet KEY : nJom2_Y_wlKb7TTWd4cEWyXsTFo8zqXL
    DEBUG | wrapper | 2021/03/08 11:25:33 | Got key from JVM: nJom2_Y_wlKb7TTWd4cEWyXsTFo8zqXL
    DEBUG | wrapperp | 2021/03/08 11:25:33 | Send a packet LOW_LOG_LEVEL : 1
    DEBUG | wrapperp | 2021/03/08 11:25:33 | Send a packet LOGFILE : /opt/SumoCollector/logs/collector.out.log
    DEBUG | wrapperp | 2021/03/08 11:25:33 | Send a packet PROPERTIES : (Property Values, Size=6514)
    DEBUG | wrapperp | 2021/03/08 11:25:33 | read a packet STOP : 1
    DEBUG | wrapper | 2021/03/08 11:25:33 | JVM requested a shutdown. (1)
    DEBUG | wrapper | 2021/03/08 11:25:33 | wrapperStopProcess(1, FALSE) called.
    DEBUG | wrapper | 2021/03/08 11:25:33 | Sending stop signal to JVM
    DEBUG | wrapperp | 2021/03/08 11:25:33 | Send a packet STOP : NULL
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager Debug: Stopped checking for control events.
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: handleBackend()
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: Stopper started
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: Thread, Wrapper-Stopper, handling the shutdown process.
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: Received a packet LOW_LOG_LEVEL : 1
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: LowLogLevel from Wrapper is 1
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: Received a packet LOGFILE : /opt/SumoCollector/logs/collector.out.log
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: shutdownJVM(1) Thread: Wrapper-Stopper
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: wait for 0 shutdown locks to be released.
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: Send a packet STOPPED : 1
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: Received a packet PROPERTIES : (Property Values, Size=6514)
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: Received a packet STOP :
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: Already stopping
    DEBUG | wrapperp | 2021/03/08 11:25:34 | read a packet STOPPED : 1
    DEBUG | wrapper | 2021/03/08 11:25:34 | JVM signaled that it was stopped.
    DEBUG | wrapper | 2021/03/08 11:25:34 | Signal trapped. Details:
    DEBUG | wrapper | 2021/03/08 11:25:34 | signal number=17 (SIGCHLD), source="unknown"
    DEBUG | wrapper | 2021/03/08 11:25:34 | Received SIGCHLD, checking JVM process status.
    DEBUG | wrapper | 2021/03/08 11:25:34 | JVM process exited with a code of 1, however the wrapper exit code was already 1.
    DEBUG | wrapper | 2021/03/08 11:25:34 | JVM exited normally.
    DEBUG | wrapperp | 2021/03/08 11:25:34 | Closing backend socket.
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: Closing backend connection.
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: Closed backend socket (Normal): java.net.SocketException: Socket closed
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: Returned from backend handler.
    INFO | jvm 2 | 2021/03/08 11:25:34 | WrapperManager Debug: WrapperManager stopped due to System Exit
    STATUS | wrapper | 2021/03/08 11:25:34 | <-- Wrapper Stopped

     

    Last edit: Gourav Garg 2021-03-08

  • Leif Mortenson

    Leif Mortenson - 2021-03-09

    Gourav
    This is being caused by a mismatch between the versions of the Wrapper binary, jar, and native library:

    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager: ERROR - The version of the Wrapper which launched this JVM is "3.5.33"
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager: while the version of the Wrapper jar file currently in use
    INFO | jvm 2 | 2021/03/08 11:25:33 | WrapperManager: is "3.5.45".

    Please make sure you update all three files from the same release.
    We also recommend that you update your shell script.

    Cheers,
    Leif

     

  • Vijit Singhal

    Vijit Singhal - 2022-05-23

    Hi Leif,

    could you please guide on how to update and make sure that all the three files are from the same release? Once we see the error and do a manual restart, everything works fine. So not sure, what needs to be changed.

     

    Last edit: Vijit Singhal 2022-05-23

  • Vijit Singhal

    Vijit Singhal - 2022-05-24

    INFO | jvm 2 | 2022/05/23 23:42:50 | WrapperManager: ERROR - The version of the Wrapper which launched this JVM is "3.5.33"
    INFO | jvm 2 | 2022/05/23 23:42:50 | WrapperManager: while the version of the Wrapper jar file currently in use
    INFO | jvm 2 | 2022/05/23 23:42:50 | WrapperManager: is "3.5.49".
    INFO | jvm 2 | 2022/05/23 23:42:50 | WrapperManager:
    INFO | jvm 2 | 2022/05/23 23:42:50 | WrapperManager: ERROR - The version of the Wrapper which launched this JVM is
    INFO | jvm 2 | 2022/05/23 23:42:50 | WrapperManager: "3.5.33-st" while the version of the native library
    INFO | jvm 2 | 2022/05/23 23:42:50 | WrapperManager: is "3.5.49-st".
    INFO | jvm 2 | 2022/05/23 23:42:50 | WrapperManager:
    STATUS | wrapper | 2022/05/23 23:42:51 | <-- Wrapper Stopped
    WARN | wrapper | 2022/05/23 23:49:10 | The version of the script (3.5.49.1) doesn't match the version of this Wrapper (3.5.49). This might cause some problems

     

  • Maxime Andrighetto

    Maxime Andrighetto - 2022-05-24

    Hello Vijit

    The first and second error say that your jar file (and also the native library) present in the lib folder are more recent than the Wrapper main executable. To fix this, you need to replace wrapper.exe (or simply 'wrapper' on Linux, Unix) with that of version 3.549.

    => it seems you did that because after you re-launched the Wrapper a few minutes later, the version of the Wrapper.exe was correctly reported as '3.5.49'. However your script version is '3.5.49.1', so you get a warning that it doesn't match with the Wrapper version.

    Did you add the tail number ".1" in the value of the wrapper.script.version variable in the script? What was the reason for editing? If you need to keep track of your own versions of the script, I recommend using a different variable to avoid such warning.

    This page describes the steps to upgrade the Wrapper and all its components (binaries, jar file, native library, script, etc.):
    https://wrapper.tanukisoftware.com/doc/english/howto-upgrade.html

    Please let me know if you have any questions.

    Best Regards,

    Maxime

     

  • Vijit Singhal

    Vijit Singhal - 2022-05-24

    @maxime-tsl thanks for the quick reply.
    Yes, we have our own version in jfrog artifactory with the version set as 3.5.49.1 and therefore we added this version in our script.

    The thing is, we have a restart script which does the restart automatically, but that fails. And doing a manual restart right after failure, the wrapper gets upgraded and works fine.

     

  • Maxime Andrighetto

    Maxime Andrighetto - 2022-05-24

    Vijit

    Please first note that the warning about the script version should not cause the Wrapper to stop.

    When you write "we have a restart script which does the restart automatically, but that fails", I understand that this is a failure in your restart script, is that correct?
    You probably need to fix your script so that it upgrades all wrapper files before attempting to restart the Wrapper.

    At the time '2022/05/23 23:49:10', I can only see one log message that you posted. I guess this is the manual restart that you're mentioning. Does the Wrapper restarts normally after that?
    If not, please post the full log and add 'wrapper.debug=TRUE' in your configuration to print the maximum amount of information.

    wrapper.script.version is only a variable that we use internally to guarantee that the script being used is from the same release as the Wrapper binary. Normally you should not need to edit that variable. If you need to edit the script, I recommend using your own variable and leave wrapper.script.version unchanged. Whenever possible, the variable should be added in the '.shconf' file, as described at the end of point 5 in the page that I sent.

    Best Regards,

    Maxime

     

  • Vijit Singhal

    Vijit Singhal - 2022-05-27

    Hi Maxime, thanks for your response. So, we identified the issue. We are spawning a child process which runs the script that stops and reinstalls our collector. But once the stop command is triggered, the child process is also killed along with the main process and the wrapper stays in stopped state.

    Is there a way to run the script as a totally independent process so that it can stop the wrapper and reinstall with the upgraded wrapper?

    Thanks

     

Log in to post a comment.