< Back to Blog

Environment Variables, Secrets, and .env Files: How I Manage API Keys Across Three Servers

TL;DR / Key Takeaways

  • Never hardcode API keys. Use .env files locally, environment variables in production, and secrets managers in CI/CD pipelines.
  • If you commit a key to git, it is already compromised. Rotate it immediately, even if the repo is private.
  • The --env-file flag and python-dotenv give you clean, consistent key injection across all environments.
  • The Predict & Profit bots ship with no hardcoded credentials and a SETUP_KEYS.md guide that walks through exactly this process.

I have three servers that touch API keys on a regular basis: my dev machine in Atlanta, a RackNerd VPS running the bots, and a second VPS handling the blog and some automation. I also have GitHub Actions in the mix. That is four places where a key can leak if I am not paying attention.

Early on I was sloppy about this. Not "committed AWS root credentials to a public repo" sloppy, but sloppy enough that I had to rotate a few keys after realizing they were sitting in a config file I had pushed somewhere. It is a rite of passage for solo developers. You do it once, it costs you an hour of panic, and you never do it again.

Here is how I manage it now.

The Core Rule

A secret that lives in source code is not a secret. It is a liability with a timer on it.

That applies to hardcoded strings in Python files, config files tracked by git, and anything that ends up in a Docker image layer. If it is in the repo, assume it is compromised eventually. Build your system so it never needs to be.

Local Development: .env Files

For local dev I use .env files and the python-dotenv library. The pattern is simple.

pip install python-dotenv

Create a .env file in the project root:

KALSHI_API_KEY_ID=your-key-id-here
KALSHI_PRIVATE_KEY_PATH=/home/steve/.ssh/kalshi_rsa_key.pem
FRED_API_KEY=your-fred-key-here
CLEVELAND_FED_TIMEOUT=10
LOG_LEVEL=DEBUG

Load it in Python:

from dotenv import load_dotenv
import os

load_dotenv()

api_key_id = os.getenv("KALSHI_API_KEY_ID")
private_key_path = os.getenv("KALSHI_PRIVATE_KEY_PATH")

if not api_key_id:
    raise ValueError("KALSHI_API_KEY_ID not set. Check your .env file.")

The explicit check matters. Silent None values cause bizarre failures downstream. I would rather the bot crash loudly at startup than silently fail when it tries to authenticate three minutes later.

The .env file goes into .gitignore immediately. Not eventually. Immediately.

# .gitignore
.env
.env.local
.env.production
*.pem
*.key

I also keep a .env.example file in the repo with all the keys present but no values. That file is committed. It documents what variables are required without exposing anything real.

KALSHI_API_KEY_ID=
KALSHI_PRIVATE_KEY_PATH=
FRED_API_KEY=
CLEVELAND_FED_TIMEOUT=
LOG_LEVEL=

New developer (or future me after a fresh clone) sees .env.example, copies it to .env, fills in the values, done.

The --env-file Flag

For Python scripts I run from the command line, especially on the VPS, I sometimes use the --env-file flag pattern instead of relying on load_dotenv() inside the script. This separates the secret loading from the application code entirely.

Python itself does not have a native --env-file flag the way Docker does, but it is easy to bolt on with argparse:

import argparse
import os
from dotenv import load_dotenv

parser = argparse.ArgumentParser()
parser.add_argument("--env-file", default=".env", help="Path to .env file")
args = parser.parse_args()

load_dotenv(dotenv_path=args.env_file)

Now I can do this:

python auto_trader.py --env-file /etc/predictandprofit/.env.prod

The production .env file lives in /etc/predictandprofit/ with permissions 600 and owned by the service user. It never touches the project directory. The project directory is cloned from git. The secrets are managed separately.

chmod 600 /etc/predictandprofit/.env.prod
chown botuser:botuser /etc/predictandprofit/.env.prod

This is the cleanest separation I have found for solo VPS deployments. Code in git. Secrets on disk with tight permissions. The two never overlap.

Production: Environment Variables in systemd

The bots run as systemd services. systemd has a clean way to inject environment variables without touching the code at all.

In the service file:

[Unit]
Description=Predict and Profit Weather Bot
After=network.target

[Service]
Type=simple
User=botuser
WorkingDirectory=/opt/predictandprofit
EnvironmentFile=/etc/predictandprofit/.env.prod
ExecStart=/opt/predictandprofit/venv/bin/python auto_trader.py
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target

The EnvironmentFile directive reads the .env file and injects all variables into the process environment before the script starts. The Python script reads them with os.getenv() the same way as always. No load_dotenv() needed in production because systemd already did the loading.

This means the same Python code runs identically in dev (where load_dotenv() reads .env) and in prod (where systemd has already injected the variables before the process starts). Clean.

GitHub Actions: Repository Secrets

The blog and some automation scripts run through GitHub Actions. For those I use GitHub's built-in secrets store.

Go to the repository, Settings, Secrets and variables, Actions. Add your secrets there. They are encrypted at rest and never appear in logs.

In the workflow file:

name: Deploy Blog

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Deploy to VPS
        env:
          VPS_SSH_KEY: ${{ secrets.VPS_SSH_KEY }}
          VPS_HOST: ${{ secrets.VPS_HOST }}
          VPS_USER: ${{ secrets.VPS_USER }}
        run: |
          echo "$VPS_SSH_KEY" > /tmp/deploy_key
          chmod 600 /tmp/deploy_key
          ssh -i /tmp/deploy_key -o StrictHostKeyChecking=no \
            $VPS_USER@$VPS_HOST "cd /opt/blog && git pull && npm run build"

The secrets are injected as environment variables into the runner. They never appear in the workflow file itself. If you accidentally log them, GitHub automatically redacts the value in the output.

One thing that catches people: secrets are not available in pull requests from forks. That is a security feature, not a bug. If your workflow fails in a fork PR and you cannot figure out why, that is probably the reason.

What Happens When You Accidentally Commit a Key

First, do not panic. Then do all of this immediately, in order.

Rotate the key first. Before you do anything else, go to the service where the key was issued, invalidate it, and generate a new one. Even if you think the repo is private. Even if you pushed it 30 seconds ago. Automated scanners hit GitHub in near-real-time. GitHub itself scans for common secret patterns. Assume it was already seen.

Remove it from the current code. Fix the code to load from environment variables, commit the fix.

Remove it from git history. This is the part people skip and it is the dangerous part. The key is still in the commit history even after you delete the file or overwrite the value. Anyone who clones the repo and runs git log -p will find it.

The tool for this is git filter-repo. It is the modern replacement for the older git filter-branch approach.

pip install git-filter-repo

# Remove all occurrences of the string from history
git filter-repo --replace-text <(echo "ACTUAL_SECRET_VALUE==>REDACTED")

Or if the key was in a specific file you want to remove entirely:

git filter-repo --path path/to/secrets_file.py --invert-paths

After running this you will need to force push:

git push origin --force --all

If the repo is on GitHub, contact GitHub support to clear their caches. If anyone else has cloned the repo, they need to re-clone. A force push does not remove the data from already-cloned copies.

The real lesson: the rotation is what protects you. The history cleanup is hygiene. Do both, but the rotation is the priority.

Multiple Environments Without Losing Your Mind

I keep separate .env files for dev and prod. Not one file with conditionals. Separate files.

/etc/predictandprofit/.env.dev
/etc/predictandprofit/.env.prod

Dev points at Kalshi's demo environment. Prod points at the live API. The KALSHI_BASE_URL variable is the main thing that differs, plus LOG_LEVEL=DEBUG in dev versus LOG_LEVEL=INFO in prod.

# dev
KALSHI_BASE_URL=https://demo-api.kalshi.co/trade-api/v2
LOG_LEVEL=DEBUG
DRY_RUN=true

# prod
KALSHI_BASE_URL=https://trading-api.kalshi.com/trade-api/v2
LOG_LEVEL=INFO
DRY_RUN=false

The DRY_RUN variable is one I check explicitly in the bot before placing any order. Even if everything else is wired up correctly, this is a second line of defense against accidentally firing real orders from a dev session.

DRY_RUN = os.getenv("DRY_RUN", "true").lower() == "true"

if DRY_RUN:
    logger.info("DRY RUN mode: order not sent")
else:
    response = kalshi_client.place_order(...)

Default is true. You have to explicitly set it to false to trade real money. That has saved me more than once.

A Few More Things Worth Knowing

Never log secrets. Obvious, but worth saying. logger.debug(f"Connecting with key {api_key}") will eventually end up in a log file that gets rotated somewhere, emailed somewhere, or displayed in a monitoring dashboard. Log key IDs (the non-secret identifier), not the keys themselves.

Audit what is in your environment. Running printenv on a production server periodically is a good habit. Make sure no keys are leaking in from parent processes, Docker environment chains, or CI/CD artifacts.

Rotate keys on a schedule, not just when you leak them. I rotate the Kalshi API key every few months. It takes five minutes and gives me fresh confidence that a key I forgot about somewhere is not still valid.

For Next.js specifically: NEXT_PUBLIC_ prefix variables are bundled into the client-side JavaScript and visible to anyone who inspects your site. Anything that should stay server-side should not have that prefix. Kalshi keys, FRED keys, anything that authenticates to an API: server-side only. .env.local is the right file for those, and it is gitignored by default in Next.js projects.

The bots ship with a SETUP_KEYS.md that walks through all of this for the specific APIs involved: Kalshi RSA key generation, FRED API key signup, and the directory structure I use on the VPS. It is the same patterns described here, applied to the actual deployment. If you buy the source code and find any of this confusing in context, that document is where to start.

Key management is not glamorous. It is the kind of thing you set up once and then forget about, which is exactly the point. Get it right once, automate the injection, and you can stop thinking about it.