Programs and Processes: From an Executable File to a Living Task

A program on disk is inert. /usr/sbin/nginx is a few megabytes of bytes sitting in a file system, consuming no CPU time and holding no memory beyond the block cache. Nothing happens to it until the kernel loads it, gives it an address space, and starts running its instructions — at that point it becomes a process, a live, tracked, resource-holding entity with its own identity. Confusing the two is a common source of production surprises: replacing a binary on disk with apt upgrade does not change what an already-running process is executing, which is exactly why a service needs an explicit restart after a library or binary update.

Permission Security closed out the previous module by asking which identity a running program acts under. This article answers the question that comes right before it: how does a file become a process in the first place, what does the kernel actually track about it, and how do you read that state with ps? The tools for continuous observation, memory accounting, scheduling priority, signaling, and job control each get their own dedicated article later in this module; here we lay the foundation they all build on.

Program vs. Process

A program is a file containing machine code, normally in ELF (Executable and Linkable Format) on Linux — for example /usr/bin/less or /usr/sbin/nginx. It can be copied, moved, or deleted without anything happening to the system's running state, because a file by itself does not execute.

A process is a running instance of a program, and the kernel maintains a data structure for each one (the kernel's internal task_struct) that records:

• a unique PID (process ID) and the PPID (parent process ID) of whoever created it;
• the credentials it runs under — real, effective, and saved UID/GID, plus any additional groups;
• its virtual memory layout — code, data, heap, stack, and mapped libraries;
• its open file descriptors — regular files, sockets, pipes, devices;
• its scheduling state — running, sleeping, stopped, or a zombie waiting to be reaped;
• signal handling dispositions — which signals it catches, ignores, or lets terminate it by default.

One program can back many simultaneous processes, each with independent state. Start sleep 300 in two separate terminals and you get two unrelated processes, each with its own PID, even though both execute the identical /usr/bin/sleep binary:

sleep 300 &
ps -C sleep -o pid,ppid,user,cmd

Sample output (your PID and username will differ):

    PID    PPID USER     CMD
   4821    3990 ali      sleep 300

The trailing & starts the command in the background instead of holding the terminal; the mechanism behind it is the subject of Foreground and Background Jobs. ps -C sleep filters by command name. Run the same line again and a second, independent PID appears — same file, two separate processes.

How the Kernel Creates a Process: fork and exec

Linux does not spawn a process out of nothing. With one exception — PID 1, created directly by the kernel at boot — every process is produced by an existing process, through a two-step mechanism:

1. fork() duplicates the calling process. The kernel creates a new process with a new PID, copies the parent's memory (using copy-on-write, so no physical memory is actually duplicated until one side writes to it), duplicates open file descriptors, and gives the child a private copy of everything else the parent held. Immediately after fork(), parent and child are running the same program, just as two separate processes.
2. execve() (commonly referred to by the family name exec) replaces the calling process's memory image with a different program. The PID is unchanged, but the code, data, and stack are all replaced by whatever the new executable specifies.

A shell demonstrates this pattern every time it runs a command: bash forks a child, and that child immediately execs the requested program (ls, sleep, nginx, whatever was typed). The shell itself is untouched; it just waits for the child to finish, unless the command was backgrounded.

You can watch this pair of steps with strace, if it is installed:

strace -f -e trace=execve,clone -- bash -c 'echo hi'

A representative, trimmed excerpt:

clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, ...) = 5391
[pid  5391] execve("/usr/bin/echo", ["echo", "hi"], 0x... /* 24 vars */) = 0
hi

clone() is the actual Linux system call that implements fork()-like behavior (fork() itself is now usually a thin wrapper over clone() with a specific flag set); execve() then swaps in /usr/bin/echo's code inside that same new PID. -f follows the forked child so its own execve shows up in the trace, and -e trace= limits output to only these two calls so the flood of other system calls does not obscure the pattern.

Threads vs. Processes: Sharing One Address Space

A thread is an independent stream of execution inside a process. Where fork() produces a new process with its own private copy of the address space, a thread is created by clone() with flags such as CLONE_VM|CLONE_THREAD|CLONE_FILES set, so it shares the creating process's memory, open file descriptors, and signal dispositions instead of copying them. Every thread of one program can therefore read and write the same variables directly — which is what makes them lightweight and fast to communicate between, and also what makes data races a concern that fully separate processes do not have.

Linux implements a thread as simply another schedulable task, which produces the one distinction that trips people up:

• what tools label a PID is really the thread group ID (TGID) — the identifier shared by every thread in the process, and the number ps prints;
• each individual thread also carries its own kernel task ID (TID), unique per thread;
• for a single-threaded process the two are equal, which is exactly why the difference is easy to overlook until a multi-threaded server appears.

Seeing the threads of a process directly:

ps -o pid,tid,%cpu,comm -L -p "$(pgrep -x nginx | head -1)"
ls /proc/$(pgrep -x nginx | head -1)/task

ps -L prints one row per thread and adds the TID column; /proc/<pid>/task holds one subdirectory per thread, each resembling a miniature per-process directory. This is the same view top -H shows interactively (covered in Process Monitoring), and it is why the l flag appears among the STAT codes below — it marks a process that is multi-threaded.

PID, PPID, and the Process Tree

Every process has exactly one PPID at any moment, forming a tree rooted at PID 1 (systemd on most modern distributions, though historically init). pstree renders this tree directly:

pstree -p $$

bash(3990)---pstree(5423)

$$ expands to the current shell's own PID, so this shows the current shell and the pstree process it just spawned as its child. On a full system, running pstree -p 1 shows the entire tree from systemd downward — services, their worker processes, login sessions, and everything each of those has spawned in turn.

When a parent process exits before its child, the child is reparented, typically to PID 1 (or, in systemd-based cgroup setups, sometimes to a nearby subreaper), rather than being left orphaned with no ancestor at all. This matters operationally: if you kill a supervising process expecting its children to die too, they may simply outlive it, silently reparented and still consuming resources.

ps -eo pid,ppid,user,stat,cmd --forest | head -20

--forest draws the same parent-child relationship as ASCII art directly inside ps output, without needing a separate tool.

Reading Process State with ps

ps is a point-in-time snapshot — unlike top or htop (covered in Process Monitoring), it prints once and exits, which makes it well suited to scripting and logging rather than live observation.

A commonly useful full listing:

ps -eo pid,ppid,user,%cpu,%mem,stat,start,etime,cmd --sort=-%cpu | head

Sample output:

    PID    PPID USER     %CPU %MEM STAT START     ELAPSED CMD
   1842       1 www-data  4.2  1.1 Sl   09:12       3:41:07 /usr/sbin/nginx: worker process
    987       1 postgres 1.8  3.4 Ssl  09:10       3:43:52 /usr/lib/postgresql/16/bin/postgres

• -e selects every process on the system, not just those attached to the current terminal.
• -o chooses exactly which fields to print, in the given order — far more predictable across systems than the historical ps aux or ps -ef shorthands, which vary subtly between the BSD and UNIX option styles that ps supports simultaneously for historical compatibility.
• --sort=-%cpu orders by CPU percentage descending; drop the - to sort ascending.
• STAT is the process state code: R running or runnable, S interruptible sleep, D uninterruptible sleep (usually waiting on I/O), T stopped, Z zombie. A trailing s marks a session leader, l marks a multi-threaded process, and + marks a foreground process group member.
• ETIME is elapsed real time since the process started, distinct from %CPU, which is actual processor time consumed as a percentage.

Finding a Specific Process

Three common approaches, each suited to a slightly different situation:

pgrep -a nginx
pidof nginx
ps -C nginx -o pid,ppid,user,cmd

pgrep -a matches by name (or a regular expression with -f against the full command line) and prints the matched command alongside each PID — the most convenient for interactive use. pidof prints only PIDs, which is convenient for scripting: kill -0 $(pidof nginx) checks whether at least one instance is alive without sending a real signal. ps -C restricts the listing to an exact command name and lets you choose output fields directly.

A Realistic Scenario: Confirming a Service Restarted Cleanly

Problem. After deploying a new build of catalog.service, you need to confirm the old process actually exited and a fresh one is running under the expected user, rather than the deployment silently having failed to replace it.

Check the current state before touching anything:

systemctl show -p MainPID -p ExecMainStartTimestamp catalog.service

MainPID=18422
ExecMainStartTimestamp=Wed 2026-07-29 08:03:11 UTC

Record this PID and start time — they are the baseline you compare against after the restart.

Restart and verify:

sudo systemctl restart catalog.service
systemctl show -p MainPID -p ExecMainStartTimestamp catalog.service
ps -o pid,ppid,user,lstart,cmd -p "$(systemctl show -p MainPID --value catalog.service)"

Expected result. MainPID changes to a new number and ExecMainStartTimestamp moves forward to the restart time; the ps line confirms the new PID is running under the service's configured user (not root, unless that is genuinely the intended account) and that its start time (lstart) matches the systemd timestamp.

Conclusion. If the PID is unchanged after restart, the unit likely failed to stop the old process (check journalctl -u catalog.service for the actual error) rather than having genuinely restarted — a case where trusting the exit code of systemctl restart alone would have hidden a real problem. This same PID-and-timestamp comparison pattern is the basis for verifying any service restart, not just this one example.

Common Mistakes

Assuming a Package Upgrade Restarted the Service

apt upgrade replaces files on disk; it does not, by itself, restart every process that has the old file mapped into memory. Check needrestart output (on Debian/Ubuntu) or compare systemctl show -p ExecMainStartTimestamp against the package's install time before assuming a fix is actually live.

Confusing PID Reuse with the Same Process Surviving

The kernel recycles PID numbers after a process exits and is reaped. A script that stores a PID and checks for it later (kill -0 $PID) can be fooled if enough time and process churn has passed for that number to be reassigned to something unrelated. Compare the process's start time or command line too, not the PID alone, in anything beyond a very short-lived check.

Killing a Parent and Expecting Children to Disappear

Terminating a supervising process does not automatically terminate its children — they are reparented, typically to PID 1, and keep running. Use a process group signal (see Signals and Process Control) or the service manager's own stop mechanism if you need the entire tree to exit.

Reading ps aux Output Without Knowing Which Dialect You Are In

ps supports BSD-style options (ps aux), UNIX-style options (ps -ef), and GNU long options simultaneously, and mixing them (ps -aux) produces a warning on some systems because -aux is parsed as UNIX-style flags, not BSD-style aux. Using explicit -o field selection avoids the ambiguity entirely and documents exactly what you intended to see.

Exercises

1. Start three background sleep processes with different durations, then write one ps -eo ... command that lists only PID, PPID, elapsed time, and command for processes named sleep, sorted by elapsed time descending.
2. Use strace -f -e trace=execve,clone on a short shell pipeline of your choice (for example bash -c 'ls | wc -l') and identify each clone/execve pair. How many child processes were created, and which commands did each one exec?
3. Pick a running service on your system (or start one, such as python3 -m http.server in the background) and write down its PID, PPID, and start time. Kill only the parent shell that launched it (if applicable) and observe whether the service process is reparented, using ps -eo pid,ppid,cmd.
4. Explain, without running anything, what state a process would show in STAT if it is currently blocked on a disk read versus waiting for terminal input, and why the difference matters when a process appears "stuck."

Verification criterion: your answers should reference the actual field names Linux tools report (PID, PPID, STAT, ETIME), not just a general description of "it looks running" or "it looks stopped."

References

• Linux man-pages: ps(1): https://man7.org/linux/man-pages/man1/ps.1.html
• Linux man-pages: fork(2): https://man7.org/linux/man-pages/man2/fork.2.html
• Linux man-pages: execve(2): https://man7.org/linux/man-pages/man2/execve.2.html
• Linux man-pages: clone(2): https://man7.org/linux/man-pages/man2/clone.2.html
• Linux man-pages: proc(5): https://man7.org/linux/man-pages/man5/proc.5.html
• Linux man-pages: pgrep(1), pkill(1): https://man7.org/linux/man-pages/man1/pgrep.1.html
• Linux man-pages: credentials(7): https://man7.org/linux/man-pages/man7/credentials.7.html
• Linux kernel documentation: process scheduling overview: https://www.kernel.org/doc/html/latest/scheduler/index.html
• strace project documentation: https://strace.io/