A Unix process coup d'etat

How to put a Unix process in sole control of the terminal (and why you shouldn't do that)

In a previous post I described how a Kotlin script can invoke an inner script and allow it to handle CTRL-C on its own, with a little help from the shell.

What if you don't want to use the shell? Originally I didn't realize the shell could solve this problem, so I started to write a Python script to help. My first attempt was along these lines:

# Incorrect code - hangs on the call to tcsetpgrp

import os
import sys

mypid = os.getpid()
os.setpgid(mypid, mypid)
os.tcsetpgrp(0, mypid)
os.execvp(sys.argv[0], sys.argv)

The idea is to take in a command (and args) and run them, after first setting up the process in a new process group so signals don't get sent to the parent (setpgid) and making that process group the foreground process group of the terminal, so it receives input/signals (tcsetpgrp).

It turns out this doesn't work. You can't call tcsetpgrp unless you are part of a process group that is currently foreground. The foreground process group can relinquish control to some other process group, but a non-foreground process group can't make itself the foreground process group.

# Almost correct code

import os
import sys

parent_pid = os.getpid()
assert os.isatty(0), "This only works if stdin is a tty"
assert os.tcgetpgrp(0) == os.getpgid(parent_pid), "This only works if we're currently in control"

# Fork to create a puppet process
pid = os.fork()
puppet_pid = pid or os.getpid()

# Install puppet terminal ruler
os.setpgid(puppet_pid, puppet_pid)
os.tcsetpgrp(0, puppet_pid)
if puppet_pid != 0:
    # Name ourselves as successors to the puppet
    os.setpgid(parent_pid, puppet_pid)

    # Regicide
    os.kill(puppet_pid, 9)  # SIGKILL

    # Coup d'etat complete
    os.execvp(sys.argv[1], sys.argv)

The idea is that we currently share control of the terminal with other processes in our process group, but we want sole control, hence the coup d'etat:

And it works! Kind of. Until you press CTRL-Z. Then things hang - even if you press CTRL-C. The problem is that CTRL-Z stops your process. Normally, this would propagate out to the shell, which would handle this gracefully, allowing you to resume with fg. But our coup d'etat means that the shell doesn't even see it.

So I guess it's not actually that helpful to pull off a coup d'etat like this. It's nice to have parent process in a different process group around to monitor your child process, so you might as well do something like this:

# Incomplete code - doesn't handle job control

import os
import sys

parent_pid = os.getpid()
assert os.isatty(0), "This only works if stdin is a tty"
assert os.tcgetpgrp(0) == os.getpgid(parent_pid), "This only works if we're currently in control"

# Fork so we can make our child the owner of the terminal in a unique process
# group, then put ourselves in the same process group and kill the child.
pid = os.fork()
child_pid = pid or os.getpid()

# To avoid a race condition, we setpgid/tcsetpgrp from both the parent and the
# child. The child will block until the parent call completes, then it will
# either be killed, or exit itself.

os.setpgid(child_pid, child_pid)
os.tcsetpgrp(0, child_pid)
if child_pid == 0:
    os.execvp(sys.argv[1], sys.argv)
else:
    # ...

But it's actually rather complex to monitor a child process correctly, and there is already software that does it right - the shell. It's better to use that, which is what my previous post describes.