What Local Large Models Can Help With: Boundaries, Scenarios, and Automation

Getting the model running and resident is just the starting point. The real question is: what should you actually do with a 35B small model on a consumer-grade GPU? The answer in this article is restrained—it cannot handle complex production logic (that is the job of Claude Code + Opus); its niche is narrow: daily, non-complex, fault-tolerant miscellaneous tasks. But in this narrow niche, it has two traits that cloud services cannot replace: offline self-sufficiency and privacy staying on-device. Once you understand this boundary, you will know which tasks are worth the effort.

1. Define Boundaries First: What Local Models Do Not Do

After installing a local model, the first reaction is often "use it to replace ChatGPT / Copilot"—then disappointment sets in. Because in pure capability, a 35B model that can run on a consumer GPU cannot beat frontier cloud models, especially in "production" tasks like multi-step reasoning, long-chain agents, and precise code.

One-sentence positioning:

Complex logic, production delivery, quality-required tasks → Claude Code + Opus; local models only handle daily, non-complex, fault-tolerant miscellaneous tasks.

Why local models cannot handle production: Small parameter models will fail in places requiring precision and multi-step reasoning, and the cost of failure in production is high (wrong code, deviated agents). The local savings in money and privacy are not enough to offset this cost. Leaving complex tasks to Opus is the first and most important rule for defining the boundary for local models.

2. The Foundation of Local Models: Offline + Privacy

Since they cannot beat cloud models in capability, why use local models? Because there are two things even the strongest cloud cannot do:

• Offline self-sufficiency: No internet, cloud inaccessible, cloud blocked, cloud price hikes or rate limits—local models keep running. They do not depend on any external service being online.
• Privacy stays on-device: Private data fed to it is not uploaded, does not enter any third-party logs or training sets.

These two points are the real reasons why "some tasks should be done locally."

The Contradiction of Cloud Fallback

A popular practice is "local first, cloud fallback": if the local model stops (e.g., gamemode stops the service while gaming), it automatically falls back to the cloud, ensuring uninterrupted use. Sounds good, but be clear-headed—

Once cloud fallback is configured, the local model is no longer offline and no longer purely private. As long as a fallback occurs once, that data goes to the cloud. Fallback and the two foundational traits of local models are inherently contradictory.

So do not blindly attach fallback to all local tasks. The honest approach is to divide into two categories by scenario:

Scenario Nature	Why Local Was Chosen	Configure Fallback?	Example
Availability Priority	Local is just the "cheap default"; offline/privacy are not hard requirements	✅ Can	openclaw daily assistant: look up info, schedule; if local stops, fallback to cloud to ensure the assistant stays online
Purity Priority	Local is chosen "because of offline/privacy"	❌ Never	daily-harvest scans sessions containing private code and keys; fallback to cloud = sending this data out

This line is exactly what is drawn in this machine's configuration: openclaw's primary = local / fallbacks = deepseek (availability priority), while llm-jobs' harvest directly exit 3 and wait for retry when local is unavailable, never going to the cloud (purity priority). The judgment criterion is simple: if "sending this task to the cloud once" violates the original intention of using local models, then it should not have fallback.

3. So What Can It Actually Help With?

In the narrow niche of "daily miscellaneous tasks + offline/privacy," the three attributes of local models unlock different categories of scenarios. Below, ✅ indicates what is truly running on this machine, 💡 indicates extendable ideas.

Zero Marginal Cost → Dare to Make Daily Miscellaneous Tasks Routine

With pay-per-use cloud models, you subconsciously "calculate before using"; with local zero cost, you dare to run low-value but high-frequency miscellaneous tasks every day.

• ✅ openclaw daily Q&A: Look up info, schedule, casual questions—no pain no matter how many times you ask. These "single-use low value but frequent" tasks can only be sustained by zero cost.
• ✅ Daily report synthesis every day: daily-report deduplicates and merges yesterday's fragmented entries into human-readable text at 09:00 every day, combined with system inspection, pending tasks, and Friday's "Weekly Perspective," writing four sections to the local email liz@liz6.com. Runs daily, multiple calls per run; paying cloud by the call would hurt.
• ✅ Weekly Perspective: An extra section in Friday's daily report—the model reads the past 7 days' completed entries, summarizes energy allocation, points out stalled items, and gives one suggestion for next week. Cross-document synthesis like this is annoying to do manually, but free for the model to do weekly.
• 💡 Extension: Automatically write a one-sentence summary for each saved article / RSS; monthly report rolling summary.

Privacy Stays on-Device → Dare to Feed Private Data

There is a category of data "you really want to process with AI, but dare not give to the cloud." Local is the only place you dare to feed.

• ✅ daily-harvest: Scans today's Claude Code / Codex / Grok sessions (including private projects, code, possible keys) + openclaw memory, extracts "what was actually done" to fill back into the daily report. This data must not go to the cloud; only local dares to feed it like this.
• 💡 Extension: Private diaries / chat logs / financial receipts for local RAG retrieval; photo OCR archiving.

Resident and Embeddable → Dare to Weld It Into Automation

The model resides on localhost:18080; any script or timer can call it at any time, without waking any cloud service or worrying about network round trips.

• ✅ llm-jobs: systemd timer calls the local model on schedule, collect → extract → deliver, full local loop (detailed in the next section).
• ✅ openclaw default model: The primary for daily agents points to local.
• ✅ System inspection noise reduction: health-snapshot only reads raw signals (failed units, backup results, ZFS water level, 24h tens of thousands of err-level logs aggregated by source, SSH authentication failures); the model is responsible for noise reduction and light/heavy sorting, producing the "🩺 System" section of the daily report. Judgments like "radicale logs INFO as err is long-term noise" or "SSH has 2FA, brute force attempts need not be reported" cannot be written into rules, but the model handles them in one sentence; when all green, only outputs "✅ All normal."
• 💡 Extension: Periodic distillation and maintenance of memory.

4. Turning Ideas Into Persistent Tasks: The llm-jobs Framework

All the automations above look the same when implemented: collect data → call local model once → write file / deliver. Instead of writing each task separately (each hand-rolling curl to call the model, each sending emails), why not extract a unified framework? ~/.local/lib/llm-jobs/ is this framework—a unified landing point for tasks that will continuously grow out of "periodically calling the local model" (daily reports, harvest, future weekly/monthly reports, memory maintenance); adding new tasks only requires writing one collection script.

Why This Layer of Abstraction

The daily report was originally an isolated script, with the only data source ~/doc/daily/, and this directory relies on two agents, Claude Code and openclaw (Miko), to "call doc-daily-log when they remember"—a best-effort approach—if anyone forgets to record, the completed work disappears entirely from the daily report (e.g., the 2026-07-03 daily report was originally empty). The problem splits into two parts:

• Coverage: Work done did not reliably enter daily. → Use daily-harvest to retrieve from real work traces (session records).
• Quality: Pure string parsing passes duplicates and uneven granularity as-is. → Use local model synthesis.

And these two are essentially "periodically calling the local model," so they are extracted into a unified framework.

Architecture Decision: systemd vs openclaw cron

Both are "timed," but the two paths differ greatly:

	systemd timer + script	openclaw cron
Task Model	Deterministic pipeline (collect → call model → deliver)	Wake up an agent session (with tool/memory/channel context)
Observability	journald → Loki, Grafana alertable	openclaw internal logs, outside the monitoring stack
Dependencies	Only depends on llama-server (also systemd)	Depends on openclaw gateway being online
Debugging	--dry-run, systemctl start, pure scripts can be versioned	Must reproduce in agent context, non-deterministic

Rule: Pipeline tasks go to systemd, agent tasks use openclaw cron. Daily reports and harvest are "read file → call model once → write file/send message" pipelines, no need for tool loops, go to systemd. openclaw cron is only reserved for scheduled tasks where "Miko really needs to think and act" (none currently).

Structure: Three-Layer Decoupling

~/.local/lib/llm-jobs/
  bin/llm-call         Unified model entry: POST localhost:18080; default closes thinking per request (--think opens it),
                       empty content treated as failure and retries with increased tokens, local down exit 3
  bin/notify-email     Unified delivery: Write to liz@liz6.com inbox via local JMAP (no SMTP/tunnel)
  bin/health-snapshot  Read-only collection of inspection signals (failed units/backup results/ZFS/journal err aggregation/SSH auth),
                       for daily-report's "system" section to feed model noise reduction
  jobs/daily-harvest   Early morning: Scan yesterday's sessions → map-reduce to extract completed items + pending → append to daily
  jobs/daily-report    Morning: Read daily + real-time inspection → synthesize four sections → local email
  README.md            Conventions + task registration form

• bin/llm-call is the only model entry — "Periodically calling local model" has only one tested implementation; model name/endpoint/fallback strategy are centrally managed. Changing models only requires changing this one place.
• bin/notify-email delivery is independent — Changing channels only affects this layer. Delivery failure is alerted by Grafana Bark, alert channel does not depend on email.
• Template unit llm-job@.service — ExecStart=%h/.local/lib/llm-jobs/jobs/%i, adding tasks does not require creating new service files.

Convention: Task scripts only call shared primitives in bin/, do not hand-roll curl to call models or send emails.

Complete Closed Loop

The harvest's scheduled instance passes --date yesterday via systemd drop-in (manual run defaults to today); at 04:00, yesterday is completely over, and the completed data is exactly what needs to be reported at 09:00. If the machine sleeps at night and misses the trigger, it doesn't matter; Persistent=true will make up for it after waking.

daily-harvest (04:00): Scans yesterday's registered local agent sessions—Claude Code ~/.claude/projects/**/*.jsonl, Codex ~/.codex/sessions/**/rollout-*.jsonl, Grok ~/.grok/sessions/*/chat_history.jsonl—each compressed into USER:/AI: plain text (removing thought and tool noise, truncated to limit) and fed to the model for map extraction of two things—"meaningful work actually completed" and "explicitly stated but unfinished," plus openclaw's own distilled memory. The source is registered by the script, no magic discovery of unknown tools; new agents need to add a finder+compact. Completed item candidates + manual entries already in today's daily are reduced together, only selecting truly completed and manually unrecorded items, outputting JSON, writing back line by line with hv_* tag, physically deleting old hv_ blocks first and recalculating each time (idempotent, can run repeatedly without stacking). Pending candidates are deduplicated against completed items and written to daily/<date>.pending (the entire chain is best-effort, failure only loses this section). Entirely local model + local write.

daily-report (09:00): Synthesizes four sections and writes to local email liz@liz6.com—① "🩺 System": health-snapshot collects real-time inspection signals, model reduces noise into a brief report (model unavailable degrades to listing only failed units, inspection never blocks sending); ② "📝 Yesterday's Completed": Reads all entries from yesterday's daily, model deduplicates and merges into human-readable text, model unavailable falls back to "original grouping" (no cloud, just no LLM synthesis), daily report will definitely be sent; ③ "🔜 Pending": Directly reads .pending file, if none, the entire section does not appear; ④ "📊 Weekly Perspective": Only on Friday, model summarizes the past 7 days' entries. Combined with date/weather (wttr.in), all sections undergo unified deterministic post-processing (empty lines between items, full-width indentation for bullet points—do not expect model layout stability), notify-email writes to inbox via local JMAP. On delivery failure, systemd failure enters Loki, Grafana Bark rule [llm-jobs] Daily report email delivery failure pushes to iPhone.

Fallback (Purity Priority Manifestation): After harvest completes, write ~/.local/state/llm-jobs/harvest-<date>.ok touch file; if local model is unavailable (stopped by gamemode), write .deferred and exit 3—do not write fake empty results, do not go to cloud, systemd marks failure into Loki. Before 09:00 report sends, if it finds no .ok touch file for yesterday, it first retries daily-harvest --date <yesterday> (model has recovered by then) before reading daily. Covers all scenarios: "machine sleeping at early morning / model stopped by gamemode." Note that the premise of this fallback chain is that .ok touch file must be honest—the 2026-07-10 incident (see "Blood and Tears" in the next section) was exactly empty results being mistakenly written as .ok, short-circuiting the entire fallback chain.

Adding a New Task

1. Write ~/.local/lib/llm-jobs/jobs/<name> (executable, supports --dry-run, only calls shared primitives in bin/)
2. Create ~/.config/systemd/user/llm-job@<name>.timer (copy daily-report, change OnCalendar)
3. systemctl --user daemon-reload && systemctl --user enable --now llm-job@<name>.timer

Operations

# Manual run (no send/write)
~/.local/lib/llm-jobs/jobs/daily-report  --dry-run
~/.local/lib/llm-jobs/jobs/daily-harvest --dry-run [--date YYYY-MM-DD]

# Manual trigger real task
systemctl --user start llm-job@daily-report.service

# View logs (journald → also enters Loki)
journalctl --user -u llm-job@daily-report.service -n 50
systemctl --user list-timers | grep llm-job

Delivery goes via local JMAP, no longer depends on WeChat bot token. notify-weixin / notify-flush are disabled; failures are now alerted by Grafana Bark.

5. Several Practical Tips for Using Local Models Well

All stepped out from these real tasks above, transferable to any "calling local model" scenario:

• Turn off thinking for daily tasks, and do it on the request side: Tasks like organizing, summarizing, categorizing do not need chain of thought; turning it off saves a lot of latency. Key is using request-level chat_template_kwargs: {"enable_thinking": false} (can override server preset), do not rely on server presets—presets are shared, and might be changed for other scenarios one day (see next blood and tear). bin/llm-call now defaults to closing thinking on the request side, use --think to explicitly open when needed.
• Make good use of long context, do not chop it up yourself: Local can open very long windows (this machine 64K–192K), just feed the entire session, entire document. harvest once increased the single session truncation limit from 10k to 40k—truncation cuts off the completion summary in the middle of the session, causing misses.
• Validate + retry structured output: When asking the model for JSON, you must tolerate parse errors and retry (bin/llm-call has built-in retry 2 times). Small models occasionally run out of format under temperature.
• Use positive extraction for judgment prompts, do not give easy exits like "output none if nothing" (Blood and Tear): harvest's map early on used negative framing ("output none if not discussion"), under temperature it would randomly judge productive sessions as "none," losing a whole day's work (same input, 1st time outputs 6 items, 2nd time "none"). Changed to "programming sessions usually have output, read full text then extract, only output none for pure consultation," then stable 5/5. Do not give LLM judgment prompts an easy exit to "output nothing," it will stably lose data in sampling.
• Do not over-merge: "Merge similar items" in the reduce stage is too aggressive, making same input sometimes 19 items, sometimes collapsing into 4 vague broad titles. Preserve granularity, leave merging to the downstream (daily report generation) step.
• HTTP 200 can also be a fake success—empty content must be treated as failure (Blood and Tear, 2026-07-10): To fix codex lag, the server preset qwen3.6-mtp-thinking was changed from enable_thinking:false to reasoning = on, causing the chain of thought to be separated into reasoning_content, batch task small max_tokens (512) all burned on thinking—returned 200 but content="", finish_reason=length. harvest treated empty output as "this session had no work," judged 17 sessions with real work as "none," and wrote .ok success touch file, morning fallback was short-circuited, daily report sent "no records yesterday." And giving thinking a budget does not win: tested 512→2048, it thinks longer and burns out anyway. Fix method two-pronged: ① llm-call treats empty content as failure (length doubles max_tokens and retries immediately, exhausted exit 3 goes to deferred); ② Batch tasks default to closing thinking on request side (previous point). Deeper lesson: Server presets for shared inference instances are an implicit contract for all callers—changing presets for one scenario requires reviewing all other callers.
• Do not expect model layout stability, use deterministic post-processing for display layer: Writing layout requirements like "empty line between each item" in prompts, small models execute unstably. Daily report's approach is model only handles content, script uniformly rearranges (empty lines between items, full-width space indentation for bullet points), no matter how model output format drifts, reader sees constant layout.
• Give enough tokens for structured output, and failure must never disguise success (Blood and Tear): A busy day's candidates can reach over a hundred (54 sessions → 163 candidates), reduce output JSON is very long, max_tokens=1500 directly truncates → JSON illegal → parse failure. Early on, parse failure was silently treated as "no new additions," and .ok success touch file was written, so morning fallback was not triggered, whole day's work silently evaporated (2026-07-06). Three iron rules: ① Give enough output tokens and retry with higher tier (6000→9000); ② Distinguish None (failure) from [] (truly no new additions) when judging empty; ③ Failure must have degradation, not discard—this machine's harvest degrades to "group by project and directly write raw candidates" (downstream daily report merges) when reduce repeatedly fails, only .deferred+exit 3 when model is truly unavailable. Any path where "failure is treated as empty result" will quietly swallow data in automation.
• Turn on --jinja for agent / tool calls: Otherwise model "can spit out toolcall tags but does not execute"—llama-server's native tool parsing (parsing output into structured tool_calls) only takes effect when --jinja is enabled. Pure conversation is unaffected, but must be on when agent uses it.

Related Documents

• local-llm.md — Deployment: VRAM budget, quantization selection, MTP, resident tidal scheduling, complete systemd configuration
• timers-and-crons.md — Complete overview of all systemd timers (including two llm-job@)
• monitoring.md — journald → Loki log aggregation (llm-jobs failures can be alerted here)