Update ChangeLog and bump version for v4.10-rc1

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>

ChangeLog.md

@@ -4,7 +4,26 @@ Change Log
 All relevant changes are documented in this file.
 
 
-[4.9][UNRELEASED]
+[4.10][] - 2025-02-23
+---------------------
+
+### Changes
+ - Add support for `cleanup:[0-3600,]script`, runs before service is
+   removed, unlike `post:script` that runs when service is stopped
+ - Improved entropy gathering in tertiary fallback of urandom plugin
+ - Reclassify `sysv` from run/task class to `service`
+
+### Fixes
+ - Fix #427: using an external getty with custom login program cause
+   Finit to use login program as the getty command
+ - Fix #428: allow `sysctl` path to be set at build time
+ - Fix possible memory leak when sourcing a service `env:file`
+ - Finit now waits until all `post:scripts` have run before returning
+   OK to `initctl <reload|runlevel>`.  This also applies to the new
+   `cleanup:scripts` introduced in this release
+
+
+[4.9][] - 2025-02-15
 --------------------
 
 > [!CAUTION]
@@ -29,7 +48,7 @@ All relevant changes are documented in this file.
    does not support SIGHUP.  This helps avoid unnecessary restarts of
    services that depend on a service that supports SIGHUP
  - Fix #417: leaking inotify watchers in the cgroup subsystem
- - Fix #422: honor `notfail` flag in `/etc/fstab`
+ - Fix #421: add support for `nofail` mount option in `/etc/fstab`
  - Fix #425: ordering bug in detecting .conf file changes to services
    and `initctl reload`, or runlevel changes
  - Fix `initctl touch` of template services, previously marking a
@@ -1740,7 +1759,7 @@ Major bug fix release.
 
 * Initial release
 
-[UNRELEASED]: https://github.com/troglobit/finit/compare/4.8...HEAD
+[UNRELEASED]: https://github.com/troglobit/finit/compare/4.9...HEAD
 [4.9]: https://github.com/troglobit/finit/compare/4.8...4.9
 [4.8]: https://github.com/troglobit/finit/compare/4.7...4.8
 [4.7]: https://github.com/troglobit/finit/compare/4.6...4.7

configure.ac

@@ -1,4 +1,4 @@
-AC_INIT([Finit], [4.9-rc1], [https://github.com/troglobit/finit/issues],
+AC_INIT([Finit], [4.10-rc1], [https://github.com/troglobit/finit/issues],
 	[finit], [https://troglobit.com/projects/finit/])
 AC_CONFIG_AUX_DIR(aux)
 AM_INIT_AUTOMAKE([1.11 foreign dist-xz subdir-objects])

doc/config.md

@@ -899,32 +899,39 @@ e.g., `halt:SIGPWR`.  To change the delay between the stop signal and
 KILL, use the option `kill:<1-60>`, e.g., `kill:10` to wait 10 seconds
 before sending `SIGKILL`.
 
-Services, including the `sysv` variant, support pre/post/ready scripts:
+Services, including the `sysv` variant, support pre/post/ready and
+cleanup scripts:
 
-  * `pre:[0-3600,]script`
-  * `post:[0-3600,]script`
-  * `ready:[0-3600,]script`
+  * `pre:[0-3600,]script` -- called before the sysv/service is stated
+  * `post:[0-3600,]script` -- called after the sysv/service has stopped
+  * `ready:[0-3600,]script` -- called when the sysv/service is ready
+  * `cleanup:[0-3600,]script` -- called when run/task/sysv/service is removed
 
 The optional number (0-3600) is the timeout before Finit kills the
 script, it defaults to the kill delay value and can be disabled by
 setting it to zero.  These scripts run as the same `@USER:GROUP` as the
-service itself, with any `env:file` sourced.  The scripts must use an
-absolute path, but are executed from the `$HOME` of the given user.  The
-scripts are not called with any argument, but get a set of environment
-variables:
+service itself, with any `env:file` sourced.  The scripts are executed
+from the `$HOME` of the given user.  The scripts are not called with any
+argument, but get a set of environment variables:
 
   * `SERVICE_IDENT=foo:1`
   * `SERVICE_NAME=foo`
   * `SERVICE_ID=1`
 
 The `post:script` is called with an additional set of environment
-variables:
+variables.  Yes, the text is correct, the naming was an accident:
 
- - `EXIT_CODE=[exited,signal]`: set to one of `exited` or `signal`
+ - `EXIT_CODE=[exited,signal,crashed]`: normal exit, signaled, or
+   crashed
  - `EXIT_STATUS=[num,SIGNAME]`: set to one of exit status code from
    the program, if it exited normally, or the signal name (`HUP`,
    `TERM`, etc.) if it exited due to signal
 
+When a run/task/sys/service is removed (disable + reload) it is first
+stopped and then removed from the runlevel.  The `post:script` always
+runs when the process has stopped, and the `cleanup:script` runs when
+the the stanza has been removed from the runlevel.
+
 > [!IMPORTANT]
 > These script actions are intended for setup, cleanup, and readiness
 > notification.  It is up to the user to ensure the scripts terminate.

man/finit.conf.5

@@ -175,13 +175,13 @@ task [s] echo "foo" | cat >/tmp/bar
 .Ed
 .It Cm sysv Oo LVLS Oc Ao COND Ac Ar /path/to/script ARGS Op -- Optional description
 Similar to
-.Cm task
+.Cm service
 is the
 .Cm sysv
 stanza, which can be used to call SysV style start/stop scripts.  The
-primary intention for this command is to be able to re-use much of
-existing setup and init scripts in Linux distributions.
-.Pp  
+primary intention for this command is to be able to re-use traditional
+SysV init scripts in Linux distributions.
+.Pp
 When entering an allowed runlevel, Finit calls
 .Cm init-script start ,
 when entering a disallowed runlevel, Finit calls
@@ -434,30 +434,42 @@ e.g.,
 .Cm kill:10
 to wait 10 seconds before sending SIGKILL.
 .Pp
-Services support the
-.Cm pre:script
-and
-.Cm post:script
-command actions as well.  These run as the same
+Services, including the
+.Cm sysv
+variant, support pre/post/ready and cleanup scripts:
+.Bl -tag -width 1n
+.It Cm pre:[0-3600,]script
+called before the sysv/service is stated
+.It Cm post:[0-3600,]script
+called after the sysv/service has stopped
+.It Cm ready:[0-3600,]script
+called when the sysv/service is ready
+.It Cm cleanup:[0-3600,]script
+called when run/task/sysv/service is removed
+.El
+.Pp
+The optional number (0-3600) is the timeout before Finit kills the
+script, it defaults to the kill delay value and can be disabled by
+setting it to zero.  These scripts run as the same
 .Cm @USER:GROUP
 as the service itself, with any
 .Cm env:file
 sourced.  The scripts must use an absolute path, but are executed from
 the
 .Cm $HOME
-of the given user.  The scripts are not called with any argument
-(currently), but both get the
-.Cm SERVICE_IDENT=foo
-environment variable set.  Here
-.Cm foo
-denotes the identity of the service, which if there are multiple
-services named
-.Cm foo ,
-may be
-.Cm foo:1 ,
-or any unique identifier specified in the .conf file.  The
+of the given user.  The scripts are not called with any argument, but
+both get a set of environment variables:
+.Pp
+.Bl -tag -width 1n -compact
+.It Cm SERVICE_IDENT=foo:1
+.It Cm SERVICE_NAME=foo
+.It Cm SERVICE_ID=1
+.El
+.Pp
+The
 .Cm post:script
-is called with an additional set of environment variables:
+is called with an additional set of environment variables, and yes the
+naming looks like it should be swapped:
 .Bl -tag -offset indent -width 1n
 .It Cm EXIT_CODE=[exited,signal,crashed]
 set to one of
@@ -471,10 +483,12 @@ set to one of exit status code from the program, if it exited normally,
 or the signal name (HUP, TERM, etc.) if it exited due to signal
 .El
 .Pp
-The scripts have a default execution time of 3 seconds before they are
-SIGKILLed, this can be adjusted using the
-.Cm kill:SEC
-modifier syntax.
+When a run/task/sys/service is removed (disable + reload) it is first
+stopped and then removed from the runlevel.  The
+.Cm post:script
+always runs when the process has stopped, and the
+.Cm cleanup:script
+runs when the the stanza has been removed from the runlevel.
 .It Cm runparts Aq DIR
 Call
 .Xr run-parts 8

plugins/procps.c

@@ -32,6 +32,10 @@
 #include "helpers.h"
 #include "plugin.h"
 
+#ifndef SYSCTL_PATH
+#define SYSCTL_PATH "/sbin/sysctl"
+#endif
+
 static void setup(void *arg)
 {
 	glob_t gl;
@@ -54,7 +58,7 @@ static void setup(void *arg)
 		for (i = 0; i < gl.gl_pathc; i++) {
 			char cmd[160];
 
-			snprintf(cmd, sizeof(cmd), "/sbin/sysctl -e -p %s >/dev/null", gl.gl_pathv[i]);
+			snprintf(cmd, sizeof(cmd), "%s -e -p %s >/dev/null", SYSCTL_PATH, gl.gl_pathv[i]);
 			run(cmd, "sysctl");
 		}
 		globfree(&gl);

plugins/urandom.c

@@ -26,6 +26,7 @@
 #include <sys/stat.h>
 #include <sys/time.h>		/* gettimeofday() */
 #include <sys/types.h>
+#include <sys/resource.h>	/* getrusage() */
 #ifdef _LIBITE_LITE
 # include <libite/lite.h>
 #else
@@ -43,13 +44,27 @@
 #endif
 
 #ifdef RANDOMSEED
+/*
+ * This is the fallback seed function, please make sure you have drivers
+ * enabling /dev/hwrng instead.
+ */
 static void fallback(FILE *fp)
 {
+	unsigned long seed;
 	struct timeval tv;
+	struct rusage ru;
 	int iter = 128;
 
 	gettimeofday(&tv, NULL);
-	srandom(tv.tv_sec % 3600);
+	getrusage(RUSAGE_SELF, &ru);
+
+	/* Mix multiple sources of "randomness" */
+	seed = tv.tv_sec ^ (tv.tv_usec << 16);
+	seed ^= ru.ru_utime.tv_sec ^ (ru.ru_utime.tv_usec << 16);
+	seed ^= ru.ru_stime.tv_sec ^ (ru.ru_stime.tv_usec << 16);
+	seed ^= getpid() << 8;
+
+	srandom(seed);
 	while (iter--) {
 		uint32_t i, prng = random();
 

src/api.c

@@ -332,6 +332,11 @@ static void api_cb(uev_t *w, void *arg, int events)
 	int sd, lvl;
 	svc_t *svc;
 
+	if (UEV_ERROR == events) {
+		dbg("%s(): api socket %d invalid.", __func__, w->fd);
+		goto error;
+	}
+
 	sd = accept(w->fd, NULL, NULL);
 	if (sd < 0) {
 		err(1, "Failed serving API request");
@@ -578,8 +583,6 @@ static void api_cb(uev_t *w, void *arg, int events)
 
 leave:
 	close(sd);
-	if (UEV_ERROR == events)
-		goto error;
 	return;
 error:
 	api_exit();

src/cgroup.c

@@ -259,6 +259,11 @@ static void cgroup_events_cb(uev_t *w, void *arg, int events)
 	ssize_t sz;
 	size_t off;
 
+	if (UEV_ERROR == events) {
+		dbg("%s(): inotify socket %d invalid.", __func__, w->fd);
+		return;
+	}
+
 	sz = read(w->fd, ev_buf, sizeof(ev_buf) - 1);
 	if (sz <= 0) {
 		err(1, "invalid inotify event");

src/conf.c

@@ -1542,6 +1542,11 @@ static int conf_iwatch_read(int fd)
 
 static void conf_cb(uev_t *w, void *arg, int events)
 {
+	if (UEV_ERROR == events) {
+		dbg("%s(): iwatch socket %d invalid.", __func__, w->fd);
+		return;
+	}
+
 	if (conf_iwatch_read(w->fd)) {
 		err(1, "invalid inotify event");
 		return;
@@ -1574,25 +1579,26 @@ int conf_monitor(void)
 	 * have or not have symlinks in place.  We need to monitor for
 	 * changes to either symlink or target.
 	 */
-	rc += iwatch_add(&iw_conf, finit_rcsd, IN_ONLYDIR);
+	rc |= iwatch_add(&iw_conf, finit_rcsd, IN_ONLYDIR);
 	snprintf(path, sizeof(path), "%s/available/", finit_rcsd);
-	rc += iwatch_add(&iw_conf, path, IN_ONLYDIR | IN_DONT_FOLLOW);
+	rc |= iwatch_add(&iw_conf, path, IN_ONLYDIR | IN_DONT_FOLLOW);
 	snprintf(path, sizeof(path), "%s/enabled/", finit_rcsd);
-	rc += iwatch_add(&iw_conf, path, IN_ONLYDIR | IN_DONT_FOLLOW);
-	rc += iwatch_add(&iw_conf, finit_conf, 0);
+	rc |= iwatch_add(&iw_conf, path, IN_ONLYDIR | IN_DONT_FOLLOW);
+	rc |= iwatch_add(&iw_conf, finit_conf, 0);
 
 	/*
 	 * Systems with /etc/default, /etc/conf.d, or similar, can also
 	 * monitor changes in env files sourced by .conf files (above)
 	 * define your own with --with-sysconfig=/path/to/envfiles
 	 */
-	rc += iwatch_add(&iw_conf, "/etc/default/", IN_ONLYDIR);
-	rc += iwatch_add(&iw_conf, "/etc/conf.d/", IN_ONLYDIR);
+	rc |= iwatch_add(&iw_conf, "/etc/default/", IN_ONLYDIR);
+	rc |= iwatch_add(&iw_conf, "/etc/conf.d/", IN_ONLYDIR);
 #ifdef FINIT_SYSCONFIG
-	rc += iwatch_add(&iw_conf, FINIT_SYSCONFIG, IN_ONLYDIR);
+	rc |= iwatch_add(&iw_conf, FINIT_SYSCONFIG, IN_ONLYDIR);
 #endif
+	rc |= conf_reload();
 
-	return rc + conf_reload();
+	return rc;
 }
 
 /*

src/exec.c

@@ -199,7 +199,6 @@ int run(char *cmd, char *log)
 int run_interactive(char *cmd, char *fmt, ...)
 {
 	int status, oldout = 1, olderr = 2;
-	char line[LINE_SIZE];
 	FILE *fp;
 
 	if (!cmd) {
@@ -244,6 +243,7 @@ int run_interactive(char *cmd, char *fmt, ...)
 
 	/* Dump any results of cmd on stderr after we've printed [ OK ] or [FAIL]  */
 	if (fp && !debug) {
+		char line[LINE_SIZE];
 		size_t len, written;
 
 		rewind(fp);
@@ -281,7 +281,7 @@ int exec_runtask(char *cmd, char *args[])
 	return execvp(_PATH_BSHELL, argv);
 }
 
-static void prepare_tty(char *tty, speed_t speed, char *procname, struct rlimit rlimit[])
+static void prepare_tty(char *tty, speed_t speed, const char *procname, struct rlimit rlimit[])
 {
 	char name[80];
 	int fd, dummy;
@@ -351,7 +351,7 @@ static int activate_console(int noclear, int nowait)
 	dprint(STDERR_FILENO, clr, strlen(clr));
 	dprint(STDERR_FILENO, msg, strlen(msg));
 	while ((rc = read(STDIN_FILENO, &ch, 1)) > 0 && ch != '\r' && ch != '\n')
-		continue;
+		;
 
 	/* On any error (likely EINTR), we avoid starting getty */
 	if (rc == -1)

src/finit.c

@@ -280,7 +280,7 @@ static int fsck(int pass)
 				sulogin(1);
 			}
 		}
-		rc += fsck_rc;
+		rc |= fsck_rc;
 	}
 
 	endmntent(fp);

src/helpers.h

@@ -141,6 +141,7 @@ static inline int ln(const char *target, const char *linkpath)
 
 	return 0;
 }
+
 static inline int dprint(int fd, const char *s, size_t len)
 {
 	size_t loop = 3;
@@ -159,7 +160,7 @@ static inline int dprint(int fd, const char *s, size_t len)
 	return rc;
 }
 
-static inline char *fgetval(char *line, const char *key, char *sep)
+static inline char *fgetval(const char *line, const char *key, char *sep)
 {
 	char *ptr, *str, *copy;
 	size_t len;

src/schedule.c

@@ -23,6 +23,7 @@
 
 #include "config.h"
 #include "finit.h"
+#include "log.h"
 #include "schedule.h"
 
 #define SC_INIT 0x494E4954	/* "INIT", see ascii(7) */
@@ -35,6 +36,7 @@ static void cb(uev_t *w, void *arg, int events)
 	struct wq *work = (struct wq *)arg;
 
 	if (UEV_ERROR == events) {
+		dbg("%s(): spurious problem with schedule work timer, restarting.", __func__);
 		uev_timer_start(w);
 		return;
 	}

src/service.c

@@ -81,7 +81,12 @@ static void service_timeout_cb(uev_t *w, void *arg, int events)
 {
 	svc_t *svc = arg;
 
-	/* Ignore any UEV_ERROR, we're a one-shot cb so just run it. */
+	if (UEV_ERROR == events) {
+		dbg("%s(): spurious problem, svc %s.", __func__, svc_ident(svc, NULL, 0));
+		uev_timer_start(w);
+		return;
+	}
+
 	if (svc->timer_cb)
 		svc->timer_cb(svc);
 }
@@ -488,28 +493,21 @@ static void source_env(svc_t *svc)
 		while (*end && !isspace(*end))
 			end++;
 		if (*end != 0) {
-			warnx("'%s=%s': not a valid identifier", key, val);
+			warnx("'%s=%s': not a valid identifier", key, value);
 			continue;	/* invalid key */
 		}
 
-		switch (wordexp(value, &we, 0)) {
-		case 0:
+		if (wordexp(value, &we, 0)) {
+			setenv(key, value, 1);
+		} else {
 			for (i = 0, *val = 0; i < we.we_wordc; i++) {
 				if (i > 0)
 					strlcat(val, " ", LINE_SIZE);
 				strlcat(val, we.we_wordv[i], LINE_SIZE);
 			}
 			setenv(key, val, 1);
-			wordfree(&we);
-			break;
-
-		case WRDE_NOSPACE:
-			wordfree(&we);
-			/* fallthrough */
-		default:
-			setenv(key, value, 1);
-			break;
 		}
+		wordfree(&we);
 	}
 
 	fclose(fp);
@@ -1524,6 +1522,7 @@ int service_register(int type, char *cfg, struct rlimit rlimit[], char *file)
 	char *id = NULL, *env = NULL, *cgroup = NULL;
 	char *pre_script = NULL, *post_script = NULL;
 	char *ready_script = NULL, *conflict = NULL;
+	char *cleanup_script = NULL;
 	char ident[MAX_IDENT_LEN];
 	char *ifstmt = NULL;
 	char *notify = NULL;
@@ -1630,6 +1629,8 @@ int service_register(int type, char *cfg, struct rlimit rlimit[], char *file)
 			post_script = arg;
 		else if (MATCH_CMD(cmd, "ready:", arg))
 			ready_script = arg;
+		else if (MATCH_CMD(cmd, "cleanup:", arg))
+			cleanup_script = arg;
 		else if (MATCH_CMD(cmd, "env:", arg))
 			env = arg;
 		/* catch both cgroup: and cgroup. handled in parse_cgroup() */
@@ -1817,6 +1818,11 @@ int service_register(int type, char *cfg, struct rlimit rlimit[], char *file)
 		parse_script(svc, "ready", ready_script, &svc->ready_tmo, svc->ready_script, sizeof(svc->ready_script));
 	else
 		memset(svc->ready_script, 0, sizeof(svc->ready_script));
+	if (cleanup_script)
+		parse_script(svc, "cleanup", cleanup_script, &svc->cleanup_tmo, svc->cleanup_script, sizeof(svc->cleanup_script));
+	else
+		memset(svc->cleanup_script, 0, sizeof(svc->cleanup_script));
+
 	if (!svc_is_tty(svc)) {
 		if (log)
 			parse_log(svc, log);
@@ -1916,8 +1922,8 @@ int service_register(int type, char *cfg, struct rlimit rlimit[], char *file)
 }
 
 /*
- * This function is called when cleaning up lingering (stopped) services
- * after a .conf reload.
+ * This function is called at the end of a runlevel change or reload
+ * command to delete all remnants of removed services.
  *
  * We need to ensure we properly stop the service before removing it,
  * including stopping any pending restart or SIGKILL timers before we
@@ -1962,10 +1968,14 @@ void service_monitor(pid_t lost, int status)
 		/* If the setup phase fails, drive svc to crashed. */
 		svc->status = status;
 		/* fallthrough */
+	case SVC_TEARDOWN_STATE:
 	case SVC_CLEANUP_STATE:
 		dbg("collected script %s(%d), normal exit: %d, signaled: %d, exit code: %d",
-		   svc->state == SVC_CLEANUP_STATE ? svc->post_script : svc->pre_script,
-		   lost, ok, sig, rc);
+		    (svc->state == SVC_TEARDOWN_STATE
+		     ? svc->post_script
+		     : (svc->state == SVC_CLEANUP_STATE
+			? svc->cleanup_script
+			: svc->pre_script)), lost, ok, sig, rc);
 		/* Kill all children in the same proess group, e.g. logit */
 		dbg("Killing lingering children in same process group ...");
 		kill(-svc->pid, SIGKILL);
@@ -1994,7 +2004,7 @@ void service_monitor(pid_t lost, int status)
 	kill(-svc->pid, SIGKILL);
 
 	/* Try removing PID file (in case service does not clean up after itself) */
-	if (svc_is_daemon(svc) || svc_is_tty(svc)) {
+	if (svc_is_daemon(svc) || svc_is_sysv(svc) || svc_is_tty(svc)) {
 		service_cleanup(svc);
 	} else if (svc_is_runtask(svc)) {
 		/* run/task should run at least once per runlevel */
@@ -2245,6 +2255,41 @@ void service_ready_script(svc_t *svc)
 	service_script_add(svc, pid, svc->ready_tmo);
 }
 
+static void service_cleanup_script(svc_t *svc)
+{
+	svc->pid = service_fork(svc);
+	if (svc->pid < 0) {
+		err(1, "Failed forking off %s cleanup:script %s", svc_ident(svc, NULL, 0), svc->cleanup_script);
+		return;
+	}
+
+	if (svc->pid == 0) {
+		char buf[CMD_SIZE];
+		char *argv[4] = {
+			"sh",
+			"-ac",
+			buf,
+			NULL
+		};
+		char *env_file;
+
+		redirect(svc);
+
+		env_file = svc_getenv(svc);
+		if (env_file)
+			snprintf(buf, sizeof(buf), ". %s; exec %s", env_file, svc->cleanup_script);
+		else
+			strlcpy(buf, svc->cleanup_script, sizeof(buf));
+
+		set_pre_post_envs(svc, "cleanup");
+		execvp(_PATH_BSHELL, argv);
+		_exit(EX_OSERR);
+	}
+
+	dbg("%s: cleanup:script %s started as PID %d", svc_ident(svc, NULL, 0), svc->cleanup_script, svc->pid);
+	service_timeout_after(svc, svc->cleanup_tmo, service_kill_script);
+}
+
 static void service_retry(svc_t *svc)
 {
 	char *restart_cnt = (char *)&svc->restart_cnt;
@@ -2349,7 +2394,7 @@ static void svc_set_state(svc_t *svc, svc_state_t new_state)
 		}
 	}
 
-	if (svc_is_daemon(svc)) {
+	if (svc_is_daemon(svc) || svc_is_sysv(svc)) {
 		char cond[MAX_COND_LEN];
 
 		snprintf(cond, sizeof(cond), "service/%s/", svc_ident(svc, NULL, 0));
@@ -2435,7 +2480,9 @@ restart:
 		if (enabled)
 			svc_set_state(svc, SVC_WAITING_STATE);
 		else {
-			if (svc_is_conflict(svc)) {
+			if (svc_is_removed(svc)) {
+				svc_set_state(svc, SVC_DEAD_STATE);
+			} else if (svc_is_conflict(svc)) {
 #if 0
 				logit(svc->nowarn ? LOG_DEBUG : LOG_INFO,
 				      "%s in conflict with %s, checking again ...",
@@ -2447,6 +2494,30 @@ restart:
 		}
 		break;
 
+	case SVC_TEARDOWN_STATE:
+		if (!svc->pid) {
+			dbg("%s: post script done.", svc_ident(svc, NULL, 0));
+			service_timeout_cancel(svc);
+
+			if (svc_is_removed(svc) && svc_has_cleanup(svc)) {
+				svc_set_state(svc, SVC_CLEANUP_STATE);
+				service_cleanup_script(svc);
+			} else
+				svc_set_state(svc, SVC_HALTED_STATE);
+		}
+		break;
+
+	case SVC_CLEANUP_STATE:
+		if (!svc->pid) {
+			dbg("%s: cleanup script done.", svc_ident(svc, NULL, 0));
+			svc_set_state(svc, SVC_HALTED_STATE);
+		}
+		break;
+
+	case SVC_DEAD_STATE:
+		/* End of the line ☠ */
+		break;
+
 	case SVC_DONE_STATE:
 		if (svc_is_changed(svc))
 			svc_set_state(svc, SVC_HALTED_STATE);
@@ -2469,8 +2540,11 @@ restart:
 			case SVC_TYPE_SYSV:
 			case SVC_TYPE_TTY:
 				if (svc_has_post(svc)) {
-					svc_set_state(svc, SVC_CLEANUP_STATE);
+					svc_set_state(svc, SVC_TEARDOWN_STATE);
 					service_post_script(svc);
+				} else if (svc_is_removed(svc) && svc_has_cleanup(svc)) {
+					svc_set_state(svc, SVC_CLEANUP_STATE);
+					service_cleanup_script(svc);
 				} else
 					svc_set_state(svc, SVC_HALTED_STATE);
 				break;
@@ -2489,11 +2563,6 @@ restart:
 		}
 		break;
 
-	case SVC_CLEANUP_STATE:
-		if (!svc->pid)
-			svc_set_state(svc, SVC_HALTED_STATE);
-		break;
-
 	case SVC_SETUP_STATE:
 		if (!svc->pid) {
 			int rc = WEXITSTATUS(svc->status);
@@ -2596,7 +2665,7 @@ restart:
 				break;
 			}
 
-			if (svc_is_runtask(svc)) { /* only run/task, not sysv here */
+			if (svc_is_runtask(svc)) {
 				svc_set_state(svc, SVC_STOPPING_STATE);
 				svc->restart_tot++;
 				svc->once++;
@@ -2818,7 +2887,7 @@ void service_notify_cb(uev_t *w, void *arg, int events)
 	ssize_t len;
 
 	if (UEV_ERROR == events) {
-		warn("Spurious problem with %s notify callback, restarting.", svc_ident(svc, NULL, 0));
+		dbg("Spurious problem with %s notify callback, restarting.", svc_ident(svc, NULL, 0));
 		uev_io_start(w);
 		return;
 	}
@@ -2876,6 +2945,7 @@ static void service_interval_cb(uev_t *w, void *arg, int events)
 
 	(void)arg;
 	if (UEV_ERROR == events) {
+		dbg("%s(): spurious problem, restarting.", __func__);
 		uev_timer_start(w);
 		return;
 	}

src/sm.c

@@ -107,6 +107,9 @@ static char *sm_status(sm_state_t state)
 	case SM_BOOTSTRAP_STATE:
 		return "bootstrap";
 
+	case SM_BOOTSTRAP_WAIT_STATE:
+		return "bootstrap/wait";
+
 	case SM_RUNNING_STATE:
 		return "running";
 
@@ -116,15 +119,20 @@ static char *sm_status(sm_state_t state)
 	case SM_RUNLEVEL_WAIT_STATE:
 		return "runlevel/wait";
 
+	case SM_RUNLEVEL_CLEAN_STATE:
+		return "runlevel/clean";
+
 	case SM_RELOAD_CHANGE_STATE:
 		return "reload/change";
 
 	case SM_RELOAD_WAIT_STATE:
 		return "reload/wait";
 
-	default:
-		return "unknown";
+	case SM_RELOAD_CLEAN_STATE:
+		return "reload/clean";
 	}
+
+	return "unknown";
 }
 
 static char sm_runlevel(int lvl)
@@ -201,7 +209,7 @@ restart:
 	switch (sm->state) {
 	case SM_BOOTSTRAP_STATE:
 		dbg("Bootstrapping all services in runlevel S from %s", finit_conf);
-		service_step_all(SVC_TYPE_RUNTASK | SVC_TYPE_SERVICE);
+		service_step_all(SVC_TYPE_RUNTASK | SVC_TYPE_SERVICE | SVC_TYPE_SYSV);
 
 		sm->state = SM_BOOTSTRAP_WAIT_STATE;
 		break;
@@ -319,6 +327,22 @@ restart:
 		sm->in_teardown = 0;
 		service_step_all(SVC_TYPE_ANY);
 
+		sm->state = SM_RUNLEVEL_CLEAN_STATE;
+		break;
+
+	case SM_RUNLEVEL_CLEAN_STATE:
+		/*
+		 * Wait for post:script or cleanup:script to be collected,
+		 * which moves the svc to HALTED or DEAD state.  We will
+		 * be called by the service_monitor() on collect.
+		 */
+		svc = svc_clean_completed();
+		if (svc) {
+			dbg("Waiting to collect post/cleanup script for %s, cmd %s(%d) ...",
+			    svc_ident(svc, NULL, 0), svc->cmd, svc->pid);
+			break;
+		}
+
 		/* Cleanup stale services */
 		svc_clean_dynamic(service_unregister);
 
@@ -372,6 +396,22 @@ restart:
 		dbg("Starting services after reconf ...");
 		service_step_all(SVC_TYPE_ANY);
 
+		sm->state = SM_RELOAD_CLEAN_STATE;
+		break;
+
+	case SM_RELOAD_CLEAN_STATE:
+		/*
+		 * Wait for post:script or cleanup:script to be collected,
+		 * which moves the svc to HALTED or DEAD state.  We will
+		 * be called by the service_monitor() on collect.
+		 */
+		svc = svc_clean_completed();
+		if (svc) {
+			dbg("Waiting to collect post/cleanup script for %s, cmd %s(%d) ...",
+			    svc_ident(svc, NULL, 0), svc->cmd, svc->pid);
+			break;
+		}
+
 		/* Cleanup stale services */
 		svc_clean_dynamic(service_unregister);
 

src/sm.h

@@ -29,9 +29,11 @@ typedef enum {
 	SM_BOOTSTRAP_WAIT_STATE,  /* Waiting for bootstrap to complete */
 	SM_RUNNING_STATE,         /* Normal state, services running */
 	SM_RUNLEVEL_CHANGE_STATE, /* A runlevel change has occurred */
-	SM_RUNLEVEL_WAIT_STATE,   /* Waiting for all stopped runlevel processes to be halted */
+	SM_RUNLEVEL_WAIT_STATE,   /* Waiting for all stopped processes to halt */
+	SM_RUNLEVEL_CLEAN_STATE,  /* Wait for post:scripts and cleanup:scripts */
 	SM_RELOAD_CHANGE_STATE,   /* A reload event has occurred */
-	SM_RELOAD_WAIT_STATE,     /* Waiting for all stopped reload processes to be halted */
+	SM_RELOAD_WAIT_STATE,     /* Waiting for all stopped processes to halt */
+	SM_RELOAD_CLEAN_STATE,    /* Wait for post:scripts and cleanup:scripts */
 } sm_state_t;
 
 typedef struct sm {

src/svc.c

@@ -348,6 +348,43 @@ svc_t *svc_stop_completed(void)
 	return NULL;
 }
 
+
+/**
+ * svc_clean_completed - Have post: and cleanup: scripts finished?
+ *
+ * Called late in runlevel and reload transitions to check if all post: and
+ * cleanup:scripts have completed.  A removed service should end up in the
+ * DEAD state, while a service with a post:script ends up in HALTED.
+ *
+ * Returns:
+ * %NULL if post: and cleanup: scripts have run, otherwise a pointer to
+ * the first found svc_t waiting for post: or cleanup:script.
+ */
+svc_t *svc_clean_completed(void)
+{
+	svc_t *svc, *iter = NULL;
+
+	for (svc = svc_iterator(&iter, 1); svc; svc = svc_iterator(&iter, 0)) {
+		if (svc_enabled(svc))
+			continue;
+
+		if (svc_is_runtask(svc) && svc->state == SVC_DONE_STATE)
+			continue;
+
+		if (svc_is_removed(svc)) {
+			if (svc->state == SVC_DEAD_STATE)
+				continue; /* any cleanup:script done */
+		} else {
+			if (svc->state == SVC_HALTED_STATE)
+				continue; /* any post:script done */
+		}
+
+		return svc;
+	}
+
+	return NULL;
+}
+
 /**
  * svc_find - Find a service object by its full path name
  * @name: Full path name, e.g., /sbin/syslogd
@@ -580,7 +617,7 @@ void svc_clean_dynamic(void (*cb)(svc_t *))
 	svc_t *svc, *iter = NULL;
 
 	for (svc = svc_iterator(&iter, 1); svc; svc = svc_iterator(&iter, 0)) {
-		if (svc->removed && cb)
+		if (svc_is_removed(svc) && cb)
 			cb(svc);
 	}
 }

src/svc.h

@@ -52,14 +52,16 @@ typedef enum {
 } svc_type_t;
 
 #define SVC_TYPE_ANY          (-1)
-#define SVC_TYPE_RESPAWN      (SVC_TYPE_SERVICE | SVC_TYPE_TTY)
-#define SVC_TYPE_RUNTASK      (SVC_TYPE_RUN | SVC_TYPE_TASK | SVC_TYPE_SYSV)
+#define SVC_TYPE_RESPAWN      (SVC_TYPE_SERVICE | SVC_TYPE_SYSV | SVC_TYPE_TTY)
+#define SVC_TYPE_RUNTASK      (SVC_TYPE_RUN | SVC_TYPE_TASK)
 
 typedef enum {
 	SVC_HALTED_STATE = 0,	/* Not allowed in runlevel, or not enabled. */
 	SVC_DONE_STATE,		/* Task/Run job has been run */
+	SVC_DEAD_STATE,		/* Process is dead and scheduled for removal */
+	SVC_CLEANUP_STATE,	/* Running cleanup: script */
+	SVC_TEARDOWN_STATE,	/* Running post: script */
 	SVC_STOPPING_STATE,	/* Waiting to collect the child process */
-	SVC_CLEANUP_STATE,	/* Running post: script */
 	SVC_SETUP_STATE,	/* Running pre: script */
 	SVC_PAUSED_STATE,	/* Condition is in flux, process SIGSTOPed */
 	SVC_WAITING_STATE,	/* Enabled but condition not satisfied */
@@ -196,13 +198,19 @@ typedef struct svc {
 	char           conflict[MAX_ARG_LEN];
 	char	       desc[MAX_STR_LEN];
 	char	       env[MAX_CMD_LEN];
+
 	char	       pre_script[MAX_CMD_LEN];
 	int	       pre_tmo;
+
 	char	       post_script[MAX_CMD_LEN];
 	int	       post_tmo;
+
 	char	       ready_script[MAX_CMD_LEN];
 	int	       ready_tmo;
 
+	char	       cleanup_script[MAX_CMD_LEN];
+	int	       cleanup_tmo;
+
 	/*
 	 * Used to forcefully kill services that won't shutdown on
 	 * termination and to delay restarts of crashing services.
@@ -240,6 +248,7 @@ void	    svc_foreach	           (int (*cb)(svc_t *));
 void        svc_foreach_type       (int types, int (*cb)(svc_t *));
 
 svc_t	   *svc_stop_completed	   (void);
+svc_t      *svc_clean_completed    (void);
 
 void	    svc_mark		   (svc_t *svc);
 void	    svc_mark_dynamic       (void);
@@ -270,6 +279,7 @@ static inline int svc_has_pidfile  (svc_t *svc) { return svc_is_daemon(svc) && s
 static inline int svc_has_pre      (svc_t *svc) { return svc->pre_script[0];  }
 static inline int svc_has_post     (svc_t *svc) { return svc->post_script[0]; }
 static inline int svc_has_ready    (svc_t *svc) { return svc->ready_script[0];}
+static inline int svc_has_cleanup  (svc_t *svc) { return svc->cleanup_script[0]; }
 
 static inline void svc_starting    (svc_t *svc) { if (svc) svc->starting = 1;       }
 static inline void svc_started     (svc_t *svc) { if (svc) svc->starting = 0;       }
@@ -390,12 +400,15 @@ static inline char *svc_status(svc_t *svc)
 			return "stopping";
 		}
 
-	case SVC_CLEANUP_STATE:
-		return "cleanup";
+	case SVC_TEARDOWN_STATE:
+		return "teardown";
 
 	case SVC_SETUP_STATE:
 		return "setup";
 
+	case SVC_CLEANUP_STATE:
+		return "cleanup";
+
 	case SVC_PAUSED_STATE:
 		return "paused";
 
@@ -408,6 +421,9 @@ static inline char *svc_status(svc_t *svc)
 	case SVC_RUNNING_STATE:
 		return "running";
 
+	case SVC_DEAD_STATE:
+		return "dead";
+
 	default:
 		return "UNKNOWN";
 	}

src/tty.c

@@ -172,7 +172,7 @@ int tty_parse_args(struct tty *tty, char *cmd, char **args)
 			passenv = 1;
 		else if (!strcmp(cmd, "rescue"))
 			tty->rescue = 1;	/* for rescue shells */
-		else if (whichp(cmd))		/* in $PATH? */
+		else if (!tty->cmd && whichp(cmd))
 			tty->cmd = cmd;
 		else
 			tty->args[tty->num++] = cmd;
@@ -229,8 +229,9 @@ int tty_parse_args(struct tty *tty, char *cmd, char **args)
 		return errno = EINVAL;
 	}
 
-	dbg("Registering %s getty on TTY %s at %s baud with term %s", tty->cmd ? "external" : "built-in",
-	   tty->dev, tty->baud ?: "0", tty->term ?: "N/A");
+	dbg("Registering %s getty on TTY %s at %s baud with term %s",
+	    tty->cmd ? "external" : "built-in",
+	    tty->dev, tty->baud ?: "0", tty->term ?: "N/A");
 
 	/* Built-in getty now comes as standalone program */
 	if (!tty->cmd) {