TL;DR / Key Takeaways

Cron runs in a stripped environment with no PATH, no virtualenv, and no working directory context, which breaks most Python scripts silently.
The >> /path/to/log 2>&1 pattern is not optional. Without it, cron errors disappear into the void and you will never know why your script stopped running.
Systemd timers are more reliable than cron for long-running or dependency-heavy scripts, but cron is fine for simple one-shot tasks if you harden the entry correctly.
The Predict & Profit trading bots run on a RackNerd VPS under systemd, not cron, because missed executions in a trading context cost money.

I have broken cron jobs so many times across so many servers that I have a mental checklist I run through before I even test anything. The problem is not cron itself. Cron works fine. The problem is that cron runs your script in an environment that looks almost nothing like the terminal session you tested it in. No virtualenv. No PATH. No working directory. No environment variables. Whatever worked perfectly at the command line will fail silently at 2am and you will not know until you check the logs, if you even have logs.

This post is the checklist. Four ways cron breaks Python scripts, four fixes, and then a decision tree for when to stop using cron entirely.

The Core Problem: Cron's Environment Is Not Your Environment

When you run a Python script in your terminal, a lot of invisible setup has already happened. Your shell profile sourced /etc/profile, your virtualenv was activated, PATH includes /usr/local/bin and wherever your Python binary lives, and your current working directory is wherever you are.

Cron does none of that. Cron starts with a near-empty environment. The default PATH is usually something like /usr/bin:/bin. That is it. No /usr/local/bin. No /home/user/.local/bin. Definitely no virtualenv.

Run this to see exactly what environment your cron jobs start with:

* * * * * env > /tmp/cron_env.txt

Add that line to your crontab, wait a minute, then read /tmp/cron_env.txt. Compare it to the output of env in your terminal. The difference is every assumption your script is silently making.

Break #1: Wrong Python Binary

The symptom is cron either can't find Python at all, or it finds system Python 3.8 when your script needs 3.12.

The fix is simple: use the full absolute path to the Python binary, not python or python3.

Find it first:

which python3
# /usr/bin/python3

# Or if you're in a virtualenv:
source /home/steve/bots/venv/bin/activate && which python
# /home/steve/bots/venv/bin/python

Then use that path directly in the crontab:

# Wrong
0 6 * * * python3 /home/steve/bots/weather_bot.py

# Right
0 6 * * * /home/steve/bots/venv/bin/python /home/steve/bots/weather_bot.py

Hardcoding the virtualenv's Python binary is the simplest way to also solve the virtualenv problem. You get the right Python version and the right installed packages in one path.

Break #2: Virtualenv Not Activated

If you use a virtualenv (and you should), your script depends on packages that only exist inside it. Cron does not activate your virtualenv. It does not know your virtualenv exists.

The wrong fix is putting source activate before the script in the crontab line. It works sometimes and fails mysteriously other times depending on the shell cron is using.

The right fix is to point directly at the virtualenv's Python binary, as shown above. That binary already knows about the virtualenv's site-packages. There is nothing to activate.

If you have a script that needs to activate the virtualenv explicitly (maybe it calls other subprocesses), wrap everything in a shell script:

#!/bin/bash
source /home/steve/bots/venv/bin/activate
cd /home/steve/bots
exec python weather_bot.py "$@"

Make it executable with chmod +x, then call the shell script from cron instead of the Python file directly.

Break #3: Wrong Working Directory

This one is subtle. Your script does something like open("config.json") and it works fine when you run it from /home/steve/bots. Cron runs it, cron's working directory is usually your home directory or root, and now config.json is not found. The script exits with a FileNotFoundError that you never see because you have no logging set up.

Two fixes. First, use os.path.dirname(os.path.abspath(__file__)) inside your script to resolve paths relative to the script file itself, not the working directory:

import os

# Bad: depends on where you launched the script from
config_path = "config.json"

# Good: always resolves relative to this file's location
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
config_path = os.path.join(BASE_DIR, "config.json")

Second, set the working directory explicitly in the crontab using cd before the Python call:

0 6 * * * cd /home/steve/bots && /home/steve/bots/venv/bin/python weather_bot.py

I do both. The cd in the crontab catches things at launch, and the BASE_DIR pattern in the code catches things when the script calls submodules or reads configs.

Break #4: Missing Environment Variables

Your script reads API keys from environment variables. Works fine in your terminal because you have ~/.bashrc setting them. Cron does not source ~/.bashrc. Your KALSHI_API_KEY is not set. The script crashes on startup and you have no idea.

Several options, in order of preference:

Option 1: Set variables directly in the crontab file. Cron lets you define environment variables at the top of the crontab before any job lines:

KALSHI_API_KEY=your_key_here
KALSHI_API_SECRET=your_secret_here
PYTHONPATH=/home/steve/bots

0 6 * * * /home/steve/bots/venv/bin/python /home/steve/bots/weather_bot.py

This works but puts secrets in a file that crontab -e can read. Fine for a personal VPS, not fine if other users can read cron files.

Option 2: Load a .env file at the top of the Python script. Use python-dotenv:

from dotenv import load_dotenv
import os

load_dotenv("/home/steve/bots/.env")

api_key = os.environ["KALSHI_API_KEY"]

The .env file lives outside the crontab and outside version control. Works in any execution context.

Option 3: Pass vars inline in the crontab:

0 6 * * * KALSHI_API_KEY=your_key /home/steve/bots/venv/bin/python /home/steve/bots/weather_bot.py

Ugly but functional for simple cases.

The `>> log 2>&1` Pattern

Every cron job you write should redirect output to a log file. No exceptions.

0 6 * * * cd /home/steve/bots && /home/steve/bots/venv/bin/python weather_bot.py >> /home/steve/logs/weather_bot.log 2>&1

What this does:

>> appends stdout to the log file. Single > would overwrite it on every run and you would lose history.
2>&1 redirects stderr to the same destination as stdout. This captures tracebacks, import errors, and anything the script prints to stderr.
Without 2>&1, Python exceptions vanish silently. This is how you get a cron job that appears to run but does nothing.

Rotate the log so it does not grow forever. Either add a logrotate config or trim it in the script itself. I usually just let the bot truncate its own log after 10,000 lines.

If you want cron to also email you on failure, set MAILTO at the top of the crontab:

MAILTO=steve@yourserver.com

On a headless VPS this is rarely set up, so the log file approach is more practical.

A Working Crontab Template

Here is a complete, hardened crontab entry that handles all four failure modes:

# Set environment
PYTHONPATH=/home/steve/bots
LOG_DIR=/home/steve/logs

# Run weather bot daily at 6:05 AM server time
5 6 * * * cd /home/steve/bots && /home/steve/bots/venv/bin/python weather_bot.py >> /home/steve/logs/weather_bot.log 2>&1

# Run econ bot at 7:00 AM and 12:00 PM
0 7,12 * * * cd /home/steve/bots && /home/steve/bots/venv/bin/python econ_bot.py >> /home/steve/logs/econ_bot.log 2>&1

Note: 5 6 runs at 6:05, not 6:00. If multiple cron jobs run at the top of the hour they compete for resources. Stagger them by a few minutes.

Check the server timezone before you commit to any schedule:

timedatectl
# or
date

Cron runs in the server's local timezone. If your server is UTC and your markets close in US Eastern, you need to account for the offset in your cron time expressions.

When to Stop Using Cron and Use Systemd Instead

Cron is fine for simple scheduled tasks. It starts the process, the process runs, the process exits. Clean and predictable.

Cron is not the right tool when:

Your script needs to keep running between scheduled tasks (it has internal loops or maintains connections)
You need automatic restart on failure
You want journalctl integration and structured logging
You have startup dependencies (network must be up, database must be reachable)
You need to control memory or CPU limits

For all of those cases, systemd is better. Two files: a .service unit and a .timer unit. The timer replaces the cron schedule, the service defines how the process runs.

Here is a minimal working example:

# /etc/systemd/system/weather-bot.service
[Unit]
Description=Predict & Profit Weather Bot
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=steve
WorkingDirectory=/home/steve/bots
EnvironmentFile=/home/steve/bots/.env
ExecStart=/home/steve/bots/venv/bin/python weather_bot.py
Restart=on-failure
RestartSec=30
StandardOutput=journal
StandardError=journal

# /etc/systemd/system/weather-bot.timer
[Unit]
Description=Run Weather Bot daily at 6:05 AM

[Timer]
OnCalendar=*-*-* 06:05:00
Persistent=true

[Install]
WantedBy=timers.target

Enable it:

sudo systemctl daemon-reload
sudo systemctl enable --now weather-bot.timer
sudo systemctl status weather-bot.timer

Persistent=true means if the server was off at 6:05 AM and comes back online at 6:30 AM, the timer fires immediately to catch up. Cron does not do that.

Check logs with:

journalctl -u weather-bot.service -f

PM2: When You Are on a Node.js Server and Cron Feels Wrong

If you are already running Node.js and have PM2 installed, it can manage Python processes too. It gives you a process monitor, auto-restart, and a web dashboard.

pm2 start /home/steve/bots/venv/bin/python --name weather-bot -- weather_bot.py
pm2 save
pm2 startup

For cron-style scheduling with PM2, use the --cron flag:

pm2 start weather_bot.py --interpreter=/home/steve/bots/venv/bin/python \
  --name weather-bot-cron \
  --cron "5 6 * * *" \
  --no-autorestart

I do not use PM2 for the trading bots. It adds a layer I do not need. But if you are already comfortable with it, it handles the PATH and environment problems more gracefully than bare cron.

Quick Decision Tree

Use cron if your script is simple, exits cleanly, runs infrequently, and you do not need restart-on-failure.

Use systemd timers if you need dependency ordering, automatic restart, resource limits, journald integration, or catch-up on missed runs.

Use PM2 if you are on a Node-heavy stack and already have it running.

Use a task scheduler like APScheduler inside your Python process if your script runs continuously and needs to schedule its own internal tasks without a second process involved.

The trading bots run under systemd. A missed execution because the server was rebooting is not acceptable when there are open positions to manage. Systemd's Restart=on-failure and Persistent=true timer behavior give me confidence that the bot will actually run when it is supposed to.

The Real Reason Cron Breaks Things

Cron is 50 years old. It was designed for a world where shell scripts ran as root on a single server and environment isolation was not a concern. Python virtualenvs did not exist. dotenv did not exist. The concept of running a data engineering pipeline inside an isolated Python environment with its own packages would have been science fiction.

The fixes in this post are all working around the same underlying issue: cron assumes you know the full context of the execution environment, and Python scripts assume they are running in a context that looks like your terminal. Those two assumptions do not meet in the middle without help.

Once you wire in the full binary path, the working directory, the environment file, and the log redirect, cron actually works reliably. I have cron jobs that have been running daily for years without a missed execution. The setup takes ten minutes. It is worth doing right the first time.

How to Run Python Scripts on a Schedule Without Cron Breaking Everything