For hcron v1.2. See here for all releases.
cron, the ubiquitous periodic scheduler, is one of the workhorses in the UNIX toolbox. There is no need to produce yet another cron ... unless it brings something new to the table. hcron does just that in a number of really useful and practical ways. For example:
- events are stored individually, each in their own file, rather than all in a single file
- events are organized hierarchically in the filesystem rather than as a table in a single file
- events are named and referenceable
- events are defined as multiple key=value settings rather than a ordered columns on a single line
- events can be activated on demand
- hcron is network oriented rather than system oriented
- support for template events to reducing and reuse settings
- support for failover events if an event cannot be spawned
- support for settings and working with variables
hcron is enterprise-ready having proven itself over many years of use in development and operational environments, by hundreds of users, with tens of thousands of events being scheduled each day.
ssh to launch events across the network. As such, it requires that the user ssh configuration be set up to not require user interaction to run (e.g., no password, no passphrase, or yes/no responses).
This is done by setting the ssh key:
and the ssh config file:
If full hostnames (i.e., which include the domain) are not used, then it may be necessary to add the following, too:
Host * applies to all hosts and may pose a security problem.
If host keys change frequently, you may also benefit from:
For more, see man page for
Each event is defined in its own file. An event file contains key=value lines, empty lines, or comment lines (starting with
hcron-event, an empty event file will be created as:
|(optional) username to use to when launching event. Must not require interaction.|
|Host on which to launch the |
|(optional) Command to run on the host (with minimal environment).|
|(optional) Comma-separated list of email addresses to send notification to on successful launch.|
|(optional) Message in email when sent.|
|Month(s) the event should be scheduled. January is month 1.|
|Days of the month the event should be scheduled. The range is variable and depends on the month.|
|Hour (0-23) of the day the event should be scheduled.|
|Minute (0-59) of the hour the event should be scheduled.|
|Day (0-6) of the week the event should be scheduled. Sunday is day 0.|
when_* fields support the standard cron wildcard (*), ranges (e.g., 1-3), steps (e.g., 0-23/2), lists (e.g., 0,2,6,23).
|(optional) :-separated list of events to execute if the event is not successfully launched.|
|(optional) :-separated list of event(s) to execute if the event is successfully launched.|
|(optional) Subject in email when sent. hcron provides a default when not specified.|
|(optional) Event is ignored if the value matches the last component of the event name.|
|(optional) Time in seconds or |
|(optional) Year (2000-2050) the event should be scheduled. Default is |
Email reminder of weekly meeting on Mondays:
Rotate logs daily at 00:05:
Augmenting command execution environment
By default, the event command is executed in a minimal environment. To augment it, set environment variables or source profile scripts. E.g., for bash,
or, to have a full login environment loaded (highly discouraged):
Event files are organized in a event tree under the user home directory:
<hcronfqdn> is the fully qualified host name of the machine running the hcron scheduler. Run "
hcron get fqdn" on the machine running the hcron-scheduler to get the correct value.
Make sure that the
<hcronfqdn> directory your events files are under matches the host name of the machine running the hcron scheduler. Otherwise, they will not be picked up by
hcron-reload and the snapshot will fail.
Event files under
events/ should be given meaningful names and may be organized using directories. Directory names will be part of the full event name. If symlinks are used, they must reference only items under the same
events/ tree and use relative paths.
Rotate logs on 3 machines:
where the event files
rotatelogs/mach3.xyz use the event definition above but with the following modifications, respectively:
Depending on the configuration (see
names_to_ignore_regexp), some filenames may be ignored.
The default setting is:
- all names starting with "
." (hidden files; gvim temporary files)
- all names ending with "
~" (commonly used for emacs temporary files)
hcron supports variables for use in event files. Some variables are provided by the system, others are defined by the user. Some are substituted early while others late.
Early substitution means that they are substituted when the event is loaded. Late substitution is done just before the event is executed.
All variables are of string type. They are specified as
name=value and processed in order
When referencing variables, they must be prefixed by an operator:
$- value of (e.g.,
#- count of (e.g.,
All system variables start with the prefix
HCRON_ and is reserved.
Early substitution variables:
|The fully qualified hostname running the hcron scheduler.|
|The full event name relative to the |
Late substitution variables:
|Local date and time of the event activation. Format: |
|Same as |
|Local time since the UNIX epoch in seconds of the event activation.|
|Same as |
|Colon-separated list of event names executed in an event chain by |
|Job id for event activation.|
|Job-group id for event activation. Shared by all jobs that are chained. Job id of initial job.|
|Job id of the parent job in a job chain. Parent job id equals job id for the initial job.|
|Local date and time of the event being queued in a job. Format: |
|Same as |
|Local time since the UNIX epoch in seconds of the event being queued in a job.|
|Same as |
|Local date and time of the event being scheduled. Format: |
|Same as |
|Local time since the UNIX epoch in seconds of the event being scheduled.|
|Same as |
|Colon-separated list of consecutive chained events to self by |
|Name of the trigger which caused the event to be scheduled and then executed. One of "clock" (matching event time), "immediate", (on hcron-scheduler start), "ondemand" (by user with |
|Origin of the trigger. E.g., "hcron-run", "hcron-scheduler", "user@hostname", an eventname.|
The activation time is when the event is activated/executed. The schedule time is when the event is scheduled based on the
when_* settings. The two are the same except when hcron is behind schedule (for whatever reason), which is rare.
Given a string, splitting will return a list of items. Joining will put a list of items together giving a string.
The split and join operators are:
<split_sep>is the separator used to split the string
<join_sep>is the separator used to join the items
The default split and join separators are ":" if not specified; the join separator is the same as the split separator if not specified. The separators may be zero or more characters.
The only exception to this rule is when working with the
HCRON_EVENT_NAME variable for which "/" is the default separator.
|"a", "b", "c"||a:b:c|
|"10", "20", "30", "40"||10_20_30_40|
|"mach1", "mach2", "mach3"||mach1:mach2:mach3|
|"10", "20", "30", "40"||10:20:30:40|
|"a", "b", "c"||a_b_c|
|"1", "2", "3", "4"||1-2-3-4|
A split operation is always accompanied by a join operation. Recall that variables are of string types only and so the intermediate "list" produced by a split operation is always joined.
Given the results of a split operation, the index operator can be used to extract items to be joined.
Indexing takes the forms:
- single index: positive or negative
- multiple: comma-separated single, range, and step indexes
When using indexing shorthand, if
<step> are not specified, the values default to 0, the number of elements, and 1, respectively, unless the step is negative in which case the
<end> values are reversed. See Python for more on indexing/slicing which is followed here.
Given the settings above:
|"a", "b", "c"||"b"||b|
|"a", "b", "c"||"a", "c"||a:c|
|"10", "20", "30", "40"||"10", "30"||10_30|
|"10", "20", "30", "40"||"40"||40|
|"1", "2", "3", "4"||"4", "3", "2", "1"||4-3-2-1|
Rather than using the value of a variable, the count can be obtained.
Give the settings above:
|"a", "b", "c"||"b"||1|
|"a", "b", "c"||"a", "c"||2|
|"10", "20", "30", "40"||"10", "30"||2|
|"10", "20", "30", "40"||"40"||1|
|"1", "2", "3", "4"||"4", "3", "2", "1"||4|
Other than what has already been described, operations cannot be combined such as:
but, instead, must be carried out with multiple steps:
In most cases, this is not much of an issue and will add only a few extra lines.
As the number of event files grows, it is sometimes helpful to be able to reuse the contents of an existing event file. This can be achieved using the include directive:
The include directive is resolved when the including event is loaded prior to early substitution.
Included event files need not be fully specified (they will be rejected if not). In fact, included files will often contain variable settings alone.
When many similar events must be defined and the differences can be handled using variables, a common event file, a template, can be used. A template is like any other event file except for the field
template_name. If, when hcron loads an event file, the
template_name field is set and its value matches the event file, hcron will recognize it as a template and ignore it. To use the template, symlinks are created which point to it.
Give the template file:
we can set up some symlinks pointing to the template file:
where three non-template events are defined (using only symlinks) with specialized
command settings so that
/var/log/syslog will be rotated. The
HCRON_EVENT_NAME is used to specialize the template to specify the log file in the
command field but could be used to set other fields, also.
When an event is scheduled, it becomes the first event in an event chain. If no other events are configured to be called afterward, then the chain remains with one event. Otherwise another event is called during the same scheduling period. Event chains are configured using
next_event (successful launch) and
failover_event (failed event launch), each of which can specify their own
The event chain is available in the
HCRON_EVENT_CHAIN variable, and the current event in
HCRON_SELF_EVENT. The maximum number of chained events is determined by the server configuration.
Given a set of event files:
one should call
two which should call
fail only if the others failed to launch. The respective events files would be (snippets only):
and the failover event:
All actions taken by the hcron scheduler are logged (e.g., to
Log entries take the form:
<timestamp>- takes the form:
<logtype>- the type of log entry (see below)
<username>- user associated with entry (optional)
<key>=<value>- zero or more unordered key+value settings
|Type||Values||Description and Format|
Alarm triggered because an execute or kill operation has timed out.
Discard user event information.
|done||Job has completed.|
|expire||Queued job expired because the time between being queued and activated is greater that |
Server is exiting.
Load user events.
Sent email for event
Sleep time between subsequent work periods.
Server has started.
|status||Jobs status information.|
Statistics for work period.
When the scheduler loads an event tree snapshot, it writes the load status of each event to a file at
Reasons for a rejected event are:
- bad definition - something is wrong with the event defintion
- template - event name matches the value in
- ignored - the event name matches the ignored event names regular expression (see
hcron list --show-status" to get this information.
Users are not allowed to use hcron by default. Instead, each user must be added to the
/etc/hcron/hcron.allow, one username per line. Comment lines must start with #.
All other scheduler configuration is done in the
/etc/hcron/hcron.conf file which takes the form of a Python dictionary. The keys of the configuration file are:
|Boolean indicating whether the local host may be targeted to run commands. The scheduler machine does not usually run commands in order to avoid overloading.|
|Boolean indicating whether events belonging to root will be run. Default is |
|Time (in seconds) allowed for a command to successfully spawn.|
|Base path for hcron events. Default is |
|Path to the log file. A relative path is anchored at |
|Maximum number of events that are spawned at once.|
|Maximum number of chained events allowed. Prevents against infinite recursion.|
|Maximum number of email notification recipients.|
|Maximum number of events a user may have.|
|Maximum size of the hcron event tree snapshot created at |
|Maximum number of events allowed for |
|Maximum number of jobs that can be queued.|
|Maximum number of symlinks to follow when resolving event file path.|
|Regular expression used to determine event names to ignore.|
|SMTP server to use for sending email notifications.|
|Delay between retry for obtaining information for |
|Number of retries to obtaining information for |
|Username used to test if credentials information is available/working.|
|Boolean indicating whether to use syslog or the log file.|