What Local Large Models Can Help With: Boundaries, Scenarios, and Automation

Getting the model running and persistent is just the starting point. The real question is: what exactly should you do with a 35B small model running on a consumer-grade GPU? The answer in this article is restrained—it cannot handle complex production logic (that is the job of Claude Code + Opus); its niche is narrow: daily, non-complex, fault-tolerant miscellaneous tasks. But in this narrow niche, it has two characteristics that cloud services cannot replace: offline autonomy and privacy staying on-device. Understanding this boundary tells you which tasks are worth the effort.

1. Define Boundaries First: What Local Models Do Not Do

After installing a local model, the first reaction is often "use it to replace ChatGPT / Copilot"—then disappointment sets in. Because in pure capability, a 35B model that can run on a consumer GPU cannot beat frontier cloud models, especially in "production" tasks like multi-step reasoning, long-chain agents, and precise coding.

One-sentence positioning:

Complex logic, production delivery, quality-required tasks → Claude Code + Opus; local models only handle daily, non-complex, fault-tolerant miscellaneous tasks.

Why local shouldn't handle production: Small parameter models will fail where precision and multi-step reasoning are required, and the cost of failure in production is high (wrong code, deviated agents). The local savings in money and privacy are not enough to offset this cost. Leaving complex tasks to Opus is the first and most important rule for defining the boundary for local models.

2. Local's Foundation: Offline + Privacy

Since it can't beat the cloud in capability, why use local? Because there are two things even the strongest cloud cannot do:

• Offline autonomy: No internet, cloud inaccessible, cloud blocked, cloud price hikes or rate limits—local keeps running. It does not depend on any external service being online.
• Privacy stays on-device: Private data fed to it is not uploaded, does not enter any third-party logs or training sets.

These two are the real reasons "some tasks should be done locally."

On the Contradiction of Cloud Fallback

A popular practice is "local first, cloud fallback": if local stops (e.g., gamemode stops the server while gaming), it automatically falls back to the cloud, ensuring uninterrupted usage. Sounds great, but be clear-headed—

Once cloud fallback is configured, local is no longer offline and no longer purely private. As long as one fallback occurs, that one instance's data goes to the cloud. Fallback and local's two core characteristics are naturally contradictory.

So don't blindly attach fallback to all local tasks. The honest approach is to split by scenario into two categories:

Scenario Nature	Why Local Was Chosen	Configure Fallback?	Example
Availability Priority	Local is just the "default for saving money"; offline/privacy are not hard requirements	✅ Yes	openclaw daily assistant: look up info, schedule; if local stops, fallback to cloud to ensure assistant stays online
Purity Priority	Local is chosen "because of offline/privacy"	❌ Never	daily-harvest scans sessions containing private code and keys; fallback to cloud = sending this data out

This line is exactly what is drawn in this machine's configuration: openclaw's primary = local / fallbacks = deepseek (availability priority), while llm-jobs' harvest directly exit 3 and wait for retry when local is unavailable, never going to the cloud (purity priority). The judgment criterion is simple: if "sending this task to the cloud once" violates the original intention of using local, then it should not have fallback.

3. So What Can It Actually Help With?

In the narrow niche of "daily miscellaneous tasks + offline/privacy," the three attributes of local models unlock different types of scenarios. Below, ✅ indicates what is truly running on this machine, 💡 indicates extendable ideas.

Zero Marginal Cost → Dare to Normalize Daily Miscellaneous Tasks

With pay-per-use cloud models, you subconsciously "calculate before using"; with local zero cost, you dare to run low-value but high-frequency miscellaneous tasks daily.

• ✅ openclaw daily Q&A: Look up info, schedule, casual questions; no pain in asking many times. These "single instance low value, but frequent" tasks are only affordable with zero cost.
• ✅ Daily report synthesis: daily-report at 09:00 every day deduplicates and merges yesterday's fragmented entries into human language, combined with system inspection, pending items, and Friday's "Weekly Perspective" to push to WeChat. Must run daily, multiple calls per day; paying per call on the cloud feels painful.
• ✅ Weekly Perspective: Friday's daily report adds a section—the model reads the past 7 days' completed entries, summarizes energy allocation, points out stalled items, and gives one suggestion for next week. Cross-document synthesis is annoying to do manually; zero cost for the model to do weekly.
• 💡 Extension: Auto-generate one-sentence summaries for each saved article / RSS; monthly rolling summary.

Privacy Stays on-Device → Dare to Feed Private Data

There is a type of data "you really want to process with AI, but dare not give to the cloud." Local is the only place you dare to feed.

• ✅ daily-harvest: Scans all Claude Code project sessions for the day (including private projects, code, possible keys) + openclaw memory, extracts "what was actually done" to fill back into the daily report. This data must not go to the cloud; only local dares to feed it like this.
• 💡 Extension: Private diary / chat logs / financial receipts for local RAG retrieval; photo OCR archiving.

Persistent Embedding → Dare to Weld It Into Automation

The model runs persistently on localhost:18080; any script or timer can call it at any time, no need to wake any cloud service, no network round-trip concerns.

• ✅ llm-jobs: systemd timer calls the local model on time, collect → extract → deliver, full local closed loop (detailed in next section).
• ✅ openclaw default model: Daily agent's primary points to local.
• ✅ System inspection noise reduction: health-snapshot only reads raw signals (failed units, backup results, ZFS water level, 24h tens of thousands of err-level logs aggregated by source, SSH auth failures); the model is responsible for noise reduction and priority sorting, producing the "🩺 System" section of the daily report. Judgments like "radicale logs INFO as err is long-term noise" or "SSH has 2FA, brute force attempts don't need reporting" cannot be written with rules; the model handles it in one sentence; when all green, only outputs "✅ All normal."
• 💡 Extension: Periodic distillation and maintenance of memory.

4. Turning Ideas Into Persistent Tasks: llm-jobs Framework

All those automations above look the same when implemented: collect data → call local model once → write file / deliver. Instead of writing each task separately (each hand-rolling curl to call the model, each sending WeChat), why not extract a unified framework? ~/.local/lib/llm-jobs/ is this framework—a unified landing point for tasks that will continuously grow in the category of "periodically calling local models" (daily reports, harvest, future weekly/monthly reports, memory maintenance); adding new tasks only requires writing one collection script.

Why This Layer of Abstraction

The daily report was originally an isolated script, with the only data source ~/doc/daily/, and this directory relies on two agents, Claude Code and openclaw (Miko), to "call doc-daily-log when they remember" in a best-effort manner—if someone forgets to record, the completed work disappears entirely from the daily report (e.g., the 2026-07-03 daily report was originally empty). The problem splits into two parts:

• Comprehensiveness: Work done didn't reliably enter daily. → Use daily-harvest to retrieve from real work traces (session records).
• Quality: Pure string parsing passes duplicates and inconsistent granularity through as is. → Use local model synthesis.

And these two are essentially "periodically calling local models," so extract a unified framework.

Architecture Decision: systemd vs openclaw cron

Both are "periodic," but the two paths differ greatly:

	systemd timer + script	openclaw cron
Task Model	Deterministic pipeline (collect → call model → deliver)	Wake an agent session (with tool/memory/channel context)
Observability	journald → Loki, can Grafana alert	openclaw internal logs, outside monitoring stack
Dependencies	Only depends on llama-server (also systemd)	Depends on openclaw gateway being online
Debugging	--dry-run, systemctl start, pure scripts can be versioned	Must reproduce in agent context, non-deterministic

Rule: Pipeline tasks go to systemd, agent tasks use openclaw cron. Daily reports and harvest are "read file → call model once → write file/send message" pipelines, no need for tool loops, go to systemd. openclaw cron is only reserved for scheduled tasks where "Miko really needs to think and act" (currently none).

Structure: Three-Layer Decoupling

~/.local/lib/llm-jobs/
  bin/llm-call         Unified model entry: POST localhost:18080; default closes thinking per request (--think opens),
                       empty content treated as failure and retries with higher token, local down exit 3
  bin/notify-weixin    Unified delivery: send to WeChat bot; --spool persists messages with expired tokens (ret=-2) to disk for queuing
  bin/health-snapshot  Read-only collection of inspection signals (failed units/backup results/ZFS/journal err aggregation/SSH auth),
                       feeds the "system" section of daily-report to model for noise reduction
  jobs/daily-harvest   Early morning: scan yesterday's sessions → map-reduce extract completed items + pending → supplement into daily
  jobs/daily-report    Morning: read daily + real-time inspection → synthesize four sections → deliver
  jobs/notify-flush    Hourly: resend WeChat messages queued in spool
  README.md            Conventions + task registration form

• bin/llm-call is the only model entry — "periodically calling local model" has only one tested implementation; model name/endpoint/fallback strategy are centrally managed. Changing model only requires changing this one place.
• bin/notify-weixin delivery is independent — Adding bark/email/multi-channel later only touches this layer.
• Template unit llm-job@.service — ExecStart=%h/.local/lib/llm-jobs/jobs/%i, adding tasks doesn't require creating new service files.

Convention: Task scripts only call shared primitives in bin/, do not each hand-roll curl to call the model or send WeChat.

Complete Closed Loop

harvest's scheduled instance passes --date yesterday via systemd drop-in (manual run still defaults to today); at 04:00 yesterday is completely over, the supplemented data is exactly what needs to be reported at 09:00. It doesn't matter if the machine sleeps at night and misses the trigger; Persistent=true will make up the run after waking.

daily-harvest (04:00): Find all Claude Code sessions from yesterday ~/.claude/projects/*/*.jsonl, compress each into USER:/AI: plain text (remove thinking and tool noise, truncate to limit) and feed to the model map to extract two types of things— "meaningful work actually completed" and "explicitly said to do but not finished," plus openclaw's own distilled memory. Completed item candidates + manual entries already in today's daily reduce, only pick truly completed and manually unrecorded ones, output JSON, write back line by line with hv_* tag, physically delete old hv_ blocks first then recalculate each time (idempotent, can run repeatedly without stacking). Pending candidates deduplicate against completed items and write to daily/<date>.pending (entire chain is best-effort, failure only loses this section). Entire process uses local model + local write.

daily-report (09:00): Synthesize four sections to push to WeChat— ① "🩺 System": health-snapshot collects real-time inspection signals, model reduces noise into brief report (if model unavailable, degrade to only listing failed units, inspection never blocks sending); ② "📝 Yesterday's Completion": read all entries from yesterday's daily, model deduplicates and merges into human language, if model unavailable fallback to "original grouping" (no cloud, just no LLM synthesis), daily report must be sent; ③ "🔜 Pending": directly read .pending file, if none, entire section does not appear; ④ "📊 Weekly Perspective": Friday only, model summarizes past 7 days' entries. Add date/weather (wttr.in), all sections undergo unified deterministic post-processing (empty lines between entries, full-width indentation for bullet points—do not expect model layout stability), notify-weixin --spool delivers.

Fallback (Purity Priority Manifestation): After harvest completes, write ~/.local/state/llm-jobs/harvest-<date>.ok marker; if local model unavailable (stopped by gamemode), write .deferred and exit 3—do not write fake empty results, do not go to cloud, systemd marks failure into Loki. Before 09:00 report sends, if it finds no .ok marker from yesterday, first make up run daily-harvest --date <yesterday> (model recovered at this time) then read daily. Covers all scenarios "machine sleeping at early morning / model stopped by gamemode." Note the premise of this fallback chain is that .ok marker must be honest—the accident on 2026-07-10 (see "Blood and Tears" in next section) was exactly empty results mistakenly written as .ok, short-circuiting the entire fallback chain.

Add a New Task

1. Write ~/.local/lib/llm-jobs/jobs/<name> (executable, supports --dry-run, only calls shared primitives in bin/)
2. Create ~/.config/systemd/user/llm-job@<name>.timer (copy daily-report, change OnCalendar)
3. systemctl --user daemon-reload && systemctl --user enable --now llm-job@<name>.timer

Operations

# Manual run (no send/write)
~/.local/lib/llm-jobs/jobs/daily-report  --dry-run
~/.local/lib/llm-jobs/jobs/daily-harvest --dry-run [--date YYYY-MM-DD]

# Manual trigger real task
systemctl --user start llm-job@daily-report.service

# View logs (journald → also into Loki)
journalctl --user -u llm-job@daily-report.service -n 50
systemctl --user list-timers | grep llm-job

If WeChat notify-weixin returns ret=-2 (context token expired), --spool caller's messages will be persisted to disk for queuing, notify-flush automatically resends hourly; sending a message to the bot on WeChat refreshes the token to accelerate delivery.

5. Several Practical Tips for Using Local Models Well

All stepped out from these real tasks above, transferable to any "call local model" scenario:

• Turn off thinking for daily tasks, and must turn off on request side: Tasks like organizing, summarizing, categorizing do not need chain of thought; turning off saves a lot of latency. Key is using request-level chat_template_kwargs: {"enable_thinking": false} (can override server preset), do not rely on server preset—preset is shared, might be changed for other scenarios one day (see next blood and tear). bin/llm-call now defaults to closing thinking on request side, --think explicitly opens when needed.
• Make good use of long context, don't chop it yourself: Local can open very long windows (this machine 64K–192K), feed entire session, entire document. harvest once raised single session truncation limit from 10k to 40k—truncation cuts off completion summary in the middle of the session, causing omissions.
• Structured output must verify + retry: Letting model output JSON requires fault-tolerant parsing, failure retry (bin/llm-call built-in retry 2 times). Small models occasionally run out of format under temp.
• Judgment prompts use positive extraction, do not give easy exit of "output none if nothing" (Blood and Tear): harvest's map early used negative framework ("output none if not discussion"), under temp would randomly judge entire productive sessions as "none," losing a whole day's work (same input 1st time outputs 6 items, 2nd time "none"). Changed to "programming sessions usually have output, read full text then extract, only pure consultation outputs none" then 5/5 stable. LLM judgment prompts do not give convenient exit of "output nothing," it will unstable lose data in sampling.
• Do not over-merge: "Merge similar items" too aggressive in reduce stage, makes same input sometimes 19 items, sometimes collapse into 4 vague big titles. Maintain granularity, leave merging to downstream (daily report generation) step.
• HTTP 200 can also be fake success—empty content must be treated as failure (Blood and Tear, 2026-07-10): To fix codex lag, changed server preset qwen3.6-mtp-thinking from enable_thinking:false to reasoning = on, thinking chain separated into reasoning_content, batch task small max_tokens (512) all burned on thinking—returned 200 but content="", finish_reason=length. harvest treated empty output as "this session has no work," 17 sessions with real work all judged "none," and wrote .ok success marker, morning fallback short-circuited, daily report sent "no records yesterday." And giving thinking budget doesn't win: tested 512→2048 it thinks longer still burns out. Fix method two-pronged: ①llm-call treats empty content as failure (length doubles max_tokens immediately retries, exhausted exit 3 goes deferred); ② batch tasks default request side close thinking (previous point). Deeper lesson: shared inference instance server preset is implicit contract for all callers—changing preset for one scenario, must review all other callers.
• Do not expect model layout stability, use deterministic post-processing for display layer: Writing layout requirements like "empty line between each item" in prompt, small model execution unstable. Daily report method is model only manages content, script uniformly rearranges (empty lines between entries, full-width space indentation for bullet points), no matter how model output format drifts, reader sees constant layout.
• Give enough tokens for structured output, and failure must never disguise as success (Blood and Tear): Busy day candidates can reach hundreds (54 sessions → 163 candidates), reduce output JSON very long, max_tokens=1500 directly truncates → JSON illegal → parsing failure. Early on treated parsing failure silently as "no new additions," also wrote .ok success marker, so morning fallback not triggered, whole day's work silently evaporated (2026-07-06). Three iron rules: ① Give enough output tokens and retry with higher tier (6000→9000); ② Judge empty must distinguish None (failure) from [] (truly no new additions); ③ Failure must have degradation not discard—this machine harvest degrades to "group by project directly write raw candidates" when reduce repeatedly fails (downstream daily report merges later), only model truly unavailable .deferred+exit 3. Any path where "failure treated as empty result" will quietly swallow data in automation.
• Agent / tool calls must open --jinja: Otherwise model "can spit toolcall tags but not execute"—llama-server's native tool parsing (parse output into structured tool_calls) only takes effect when --jinja is opened. Pure conversation unaffected, but when agent uses must open.

Related Documents

• local-llm.md — Deployment: VRAM budget, quantization selection, MTP, persistent tidal scheduling, complete systemd config
• timers-and-crons.md — All systemd timer overview (including llm-job@ two)
• monitoring.md — journald → Loki log aggregation (llm-jobs failures can alert here)