← All writing

No disconnection worries! 3 ways to run programs in the background on remote servers: `nohup`, `tmux` and `screen`

In data analysis or machine learning projects, it is often necessary to run long, computationally intensive tasks on remote servers. Connecting to a remote server via SSH is a common way of doing this. But how to ensure that the program on the remote server can continue to run after disconnecting the SSH connection? This article details three methods: nohup, tmux and screen.

阅读中文版 →

No disconnection worries! 3 ways to run programs in the background on a remote server:nohuptmuxandscreen

Uninterrupted Remote Program Execution: 3 Methods

In data analysis or machine learning projects, it is often necessary to run long, computationally intensive tasks on remote servers. Connecting to a remote server via SSH is a common way of doing this. But how to ensure that the program on the remote server can continue to run after disconnecting the SSH connection? This article details three methods:nohuptmuxandscreen

usenohupcommand

start

  1. Connect to remote machine: Execute the following command in the local terminal.

    1
    ssh username@remote-server-address
  2. Start background program: Executed on the remote machine.

    1
    nohup your-command-to-run-the-program &

For example:

1
nohup python train_model.py &

This method redirects the program's output to a file callednohup.outin the file.

end

Execute the following commands on the remote machine.

  1. **Find the Process ID (PID) of a program**:

    1
    ps aux | grep your-command-to-run-the-program
  2. end process

    1
    kill -9 PID
  3. Output meaning:
    When you execute the following command:

1
nohup python train_model.py &

You may see output like this:

1
2
[1] 337082
nohup: ignoring input and appending output to 'nohup.out'

Here,

  • 1 337082 Indicates that the task is now running in the background with a PID (process ID) of 337082.
    • 337082: This is the process ID (PID) for this background job. You can use this PID to monitor or kill the process.
  • nohup: ignoring input and appending output to ‘nohup.out’ Indicates that the process will now ignore any input and append output to nohup.out file.
    • This is nohup The standard message of the command has the following meaning:
      • ignoring input: when you use nohup command, it will not receive any data entered from the terminal. This means that once you use nohup After starting a program, you can no longer provide any interactive input to it (unless the program has other input methods).
      • appending output to 'nohup.out': By default,nohup will redirect the program's output to a file called nohup.out in the file. So, if your program produces any output during execution (such as print statements), this output will be written to nohup.out in the file. If the file did not exist before,nohup will automatically create it; if the file already exists,nohup New output will be appended to the end of the file.
      • If you want to see the output of your program, you can use cat or tail command to view nohup.out The contents of the file. For example, use tail -f nohup.out The content at the end of the file can be viewed in real time, which is useful for monitoring the running status of the program.

usetmux

start

  1. **Installationtmux**: Execute the following command on the remote machine.
  • Ubuntu/Debian
    1
    sudo apt-get install tmux # Ubuntu/Debian
  • MacOS
    1
    brew install tmux
  1. Connect to remote machine: Execute in local terminal.

    1
    ssh username@remote-server-address
  2. Start a new tmux session: Executed on the remote machine.

    1
    tmux

Then run your program inside the tmux session.

  1. Disconnect session: Press on the remote machine Ctrl+b d

end

Execute the following commands on the remote machine.

  1. Reconnect to tmux session

    1
    tmux attach
  2. End programs and sessions: useCtrl+cEnd the program and pressCtrl+bPress againxEnd the tmux session.

usescreen

start

  1. **Installationscreen**: Execute in local terminal.
  • Ubuntu/Debian
    1
    sudo apt-get install screen # Ubuntu/Debian
  • MacOS
    1
    brew install screen
  1. Connect to remote machine: Execute in local terminal.

    1
    ssh username@remote-server-address
  2. Start a new screen session: Executed on the remote machine.

    1
    screen

Then run your program inside the screen session.

  1. Disconnect session: Press on the remote machine Ctrl+a d

end

Execute the following commands on the remote machine.

  1. Reconnect to screen session

    1
    screen -r
  2. End programs and sessions: useCtrl+cEnd the program and pressCtrl+aPress againkEnd the screen session.

Special case: macOS users

macOS users can use Homebrew to installtmuxorscreen

  • Installationtmux

    1
    brew install tmux
  • Installationscreen

    1
    brew install screen