---
title: "Your Local AI Harness Can Still Be a Landlord"
url: https://stacklist.com/card/b830a195-e4ac-4fe5-879d-ca3df6c6e9bf
source_url: "https://x.com/alex_verem/status/2096994769503502666?s=12"
stack: https://stacklist.com/c/technology/stack/1659549d-373d-4391-ba12-5a14d40c19ed
summary: "Local AI harnesses can create a new form of lock-in despite appearing to offer sovereignty, as the harness defaults and configurations determine what an agent can access and do rather than the model weights themselves. The article advocates for Meta AI's Rule of Two principle, which limits agents to holding at most two of three capabilities—untrusted input, private data access, and ability to change things or communicate externally—within a single session."
tags: "local-ai, agent-security, model-autonomy, landlord-problem, ai-governance, harness-design"
key_entities: "Alex Veremeyenko (person), Simon Willison (person), Anthropic (organization), Meta AI (organization), lethal trifecta (concept), Rule of Two (concept), harness (concept)"
classification: "analysis"
content_hash: "sha256:bdeeba5a2d47f9c10bd93f00eb2937b456e2d826f4baa17d35aa95ee25896968"
acp_version: "0.2"
token_counts_approximate: 3222
visibility: public
agent_accessible: true
status: "final"
---

# Your Local AI Harness Can Still Be a Landlord

Alex Veremeyenko @alex_verem Your Local AI Harness Can Still Be a Landlord Let me guess how this went for you. You got tired of paying a cloud lab every month for a chatbot that kept changing its rules, so you bought a decent GPU, pulled down some open weights, and wired up one of the local agent harnesses everyone keeps talking about. Nothing leaves your network anymore, and it feels like you finally own the thing. I want to poke at that feeling a little, because I think a lot of people running local models have swapped one landlord for another without noticing. A cloud agent locks you in with a subscription, and at least you can cancel a subscription. A local agent you built with no limits locks you in with the machine you trusted, and there is no cancel button for a setup you wrote yourself. The security researcher Simon Willison gave the dangerous version of this a name in June 2025. He pointed out that once an agent can read your private data, gets exposed to content you don't control, and has some way to send information out into the world, a single poisoned input is enough to turn it against you. He called that combination the lethal trifecta, and the uncomfortable part is that you grant all three of those on purpose, usually because whatever task you had in mind seemed to need them. Anthropic's engineering team made basically the same argument from the builder's side in December 2024, and they weren't even talking about security. Their advice was to find the simplest system that does the job, and to accept that sometimes the simplest system is no agent at all, because every layer of autonomy you add costs you latency and money and gives mistakes more room to pile up on top of each other. So here is the frame I'd like you to carry through the rest of this. A harness is the loop that wraps around the model. It decides what the model is allowed to call, where it can write, how long it can keep going, and what finally makes it quit. Your weights don't decide any of that, no matter whose disk they're sitting on. The loop does, and if sovereignty lives anywhere in your setup, it lives in the things that loop refuses to do. The terms you never wrote Whoever writes the terms owns the place. That is really the whole job description of a landlord, and it's worth holding onto because it makes the rest of this obvious. I spend most weeks aiming that exact complaint at the cloud labs, and I still think the aim is fair. Their terms live in a pricing page and a data policy you never got to edit. So you moved out. The weights are on your disk, inference runs on your own card, and no packets leave your router. Then you left the harness defaults exactly where the installer put them, added every skill that looked even slightly useful, pointed the whole thing at your home directory, and let it run overnight because that felt like the future. The defaults wrote the terms this time around, and you're living inside them. You are a tenant on hardware you paid for. Here is the distinction I'd ask you to sit with for a second. Running a model locally answers one question, which is where the model runs. It says nothing at all about what the agent can reach or what stops it once it gets going. Ownership lives in those two questions, and moving the weights home doesn't touch either of them. Meta AI took Willison's three ingredients and turned them into an actual ceiling in October 2025. Their Rule of Two says that an agent, within a single session, should hold at most two of the following three things. Untrusted input, private data, and the ability to change something or talk to the outside world. If a job genuinely needs all three, the agent runs under supervision, which means a human looks at what it wants to do and approves it before it happens. It's worth checking your own setup against that, and it only takes a minute. If your agent reads your inbox or browses the open web, that's untrusted input. If it has your files, that's private data. If it can run a shell command or send a message on your behalf, that's external action. If you said yes to all three, you're running the exact configuration Meta says needs a human in the loop, and you gave it none, because nobody was around to make you. A cloud lab would at least have charged you for the privilege. So here is what I think separates someone who owns their stack from someone who just hosts it. They can tell you four things about their agent without opening a terminal, meaning its one task, which tools it can and cannot touch, the ceiling on what a single run can consume, and the thing that stops it. If you can't name all four off the top of your head, the harness is running you. One task and not a personality Scope is the first limit, and the general-purpose assistant is the trap most people walk into first, because it's the one that looks the most impressive in a demo. An agent with one job has one failure you can describe. It sorts your invoices badly, or it doesn't. An agent with thirty jobs has thirty different ways to fail, plus all the combinations you never thought about. Picture the invoice skill reading a PDF that happens to contain an instruction for the browser skill, which opens a page that the messaging skill then forwards to someone. None of those three skills is dangerous on its own. You built the chain the moment you installed all three into the same brain. The reason the pile grows is that adding a tool at home costs you nothing. There's no invoice, no rate limit, and no approval screen asking whether you're sure. In the cloud, price was quietly doing some gatekeeping on your behalf, and when you went local, you removed the gatekeeper and kept installing. The fix is honestly kind of boring. Run one agent per job, give each of them its own workspace, and start every skill list empty so it only grows when the task forces it to. To take one local harness as an example, OpenClaw lets each agent carry its own skill allowlist, and an empty list is a perfectly valid configuration. Start there and make every addition justify itself. And please don't build the fleet on day one. One narrow agent whose worst day you can describe in a sentence is worth more than a personal Jarvis whose worst day you can't describe at all. One toolset, and deny wins Reach is the real sovereignty question, and it's the one that separates an annoying agent from a dangerous one. A bad output is a paragraph you delete. A bad action is a file you can't get back. So write down what the agent can reach, write down what it can never reach, and make that second list the stricter rule. Allow lists describe your intent. Deny lists describe the fence. If the two disagree, the fence should win at every layer, and nothing further down the chain should be able to hand a denied tool back to the model. Some local harnesses already ship in this shape, and it helps to know what to look for. OpenClaw's tool policy, going by its own documentation, applies deny over allow at every level, and each level is only able to restrict things further, never loosen them. Its sandbox stays switched off until you turn it on, which is the landlord's default, doing exactly what defaults do. Two smaller details in that design tell you the authors were thinking about the right problem. An agent whose allowlist works out to zero callable tools refuses to start at all, rather than quietly running as a chatbot and pretending everything is fine. And a command typed into the chat can't override a deny that was written in config, because the person typing into the chat window counts as untrusted input too. That second list is the standard you're aiming for. The things you wrote down that it can never touch. One budget, and local means it's compute, not cash A loop with no ceiling keeps consuming until something outside of it steps in and stops it. In the cloud, that something is your credit card. It's a terrible kill switch, but at least it fires eventually. At home there is no bill. Instead, you get a fan running at full speed at three in the morning and an agent that has been retrying the same failed search since midnight, because nothing ever told it the number of attempts had a limit. The budget you actually own is measured in wall-clock time, CPU cores, memory, how many child agents a single run is allowed to spawn, and how often the thing is allowed to wake up on its own without you asking. Every one of those is a ceiling you either set on purpose or inherited by accident. Two examples from the same harness make this concrete. Its sandbox can cap CPUs, memory, and process count per container, and its subagent spawning has both a depth limit and a limit on children per agent. Its heartbeat feature, the scheduled wake-up that lets an agent check in without being prompted, runs a full model turn every single time it fires, and the documentation says plainly that shorter intervals burn more tokens. That's a tax you signed up for the moment you set the interval. Set the ceiling before the first overnight run. The morning after is a bad time to find out what the defaults were. One kill switch, and a prompt isn't one A stop rule has to live somewhere the model can't argue with it, and this is the part that trips up a lot of people coming over from chat interfaces, because in a chat window the words always seemed to be enough. Let me show you what it looks like when that assumption fails. In February 2026, an OpenClaw user filed issue #9912 on the project's GitHub. Their system prompt told the model, under a heading marked critical, to stop calling tools after the first successful result. The logs they posted show the tool coming back in 8 milliseconds, the model going quiet for about five minutes, and then the same tool being called again. The prompt said STOP in capital letters. The model treated it as a suggestion. The user reported that they couldn't find any config-level cap on turns or tool calls, and asked for one to be added. The harness in that story was self-hosted, and the model behind it was served through a cloud API, and I'd argue that detail is what makes the example travel. Swap in local weights and nothing about the mechanism changes. A prompt is just input. The model weighs it against everything else sitting in its context, including whatever a poisoned webpage told it thirty seconds ago. If the model can read the kill switch, a stranger can talk the model out of it. Real stops live in the harness, and they tend to look unglamorous. A wall-clock timeout on every run. A session that resets on a schedule, or after some idle time, whether or not the agent thinks it's finished. A cap on how many runs can happen at once. And some way to print the policy that's actually in effect, so you're reading what's enforced rather than what you remember writing six weeks ago. Harnesses vary in which of these they ship, and that's fine. Your job is knowing which ones yours has, and which ones you've been quietly substituting with a sentence in a prompt. Any weekend project can call a model inside a while-true loop. The stop is what takes real work, and a working demo makes it very easy to skip. The limits are the point Once this clicks, the order of your questions flips. You read the limits first, and the capabilities can wait. The deed to your setup is the config file, the document that says this task, these tools, this ceiling, this stop. Anyone can read it. Hand it to a friend, and they'll know what your agent is without watching a single demo. The GPU under your desk proves where the work happens. The deed proves who set the terms. And the deed outlives the harness, which matters more than it sounds. Rename the tool, swap the model, move the whole setup to a different machine, and the four limits still read the same, because you wrote them in terms of the job rather than the product. Fewer verbs don't give you a dumber agent, either. It stays every bit as good at the one job you handed it. The difference is that you can read the job, put a boundary around the failure, and fix it on a Tuesday afternoon instead of dreading it. Sovereignty is not having more capabilities on your own machine. It's choosing the limits those capabilities cannot cross. Own the weights, fine. Then own the no's. The landlord is whoever writes the terms, so write them. Thirty tools and a no-stop rule is a lease you signed with yourself. I curate the best AI projects actually solving real problems. Ecology. Medicine. Agriculture. Not chatbot tricks. Follow for the AI that matters 👇 https://x.com/alex_verem 4:11 PM · Sep 7, 2026 18.5K Views 4 4 30 57