Should Your AI Assistant Run Shell Commands? I Went to Check My Sandbox First

Here is the entire feature request, in the user's own words:

"Let support executing command line using script. … The permission screen should show the commands being executed."

Reasonable. Savant scripts are small JavaScript programs the AI writes and runs for you. They already read files, query your local databases, and call other tools, all behind a permission manifest you approve once. The user has pandoc installed. They want a script to turn a Markdown file into a PDF, and they're offering to look at a screen that tells them what will run.

Two sentences. An afternoon of work.

That request has been open since the 21st of April, and my answer for nearly three months has been a document.

Why I kept not shipping it

A Savant script declares what it needs, you approve it, and the runtime checks every call against what you approved. There are five of those checks in pkg/scripting/sandbox.go: file paths, URLs, databases, agents, and tool names.

Each one is a door I built on purpose, and each was supposed to be narrow. fetch takes a URL and the check compares its host against the domains you allowed. fs.readFile takes a path and the check refuses anything outside the script's own folder and the paths it declared. db.execute gets the databases you created and nothing else.

Now put a general cmd.run() next to them. curl is fetch without the domain list. cat and cp are the file API without the path check. sqlite3 is the database API pointed at any file on disk. And if you allow python, or node, or bash, you haven't allowed a tool at all, you've allowed anything that can be typed.

None of that is my insight. Deno's docs have said it for years: --allow-run escalates to every other permission, so it's effectively --allow-all. It's a known result, and I treated it as the reason to keep the door shut.

So I wrote threat models. Prompt injection, where a script reads a web page or an email and the content carries instructions aimed at the model. Approval fatigue, where you click Approve without reading, because that's what every human does with permission dialogs and has done since the first one shipped. Both are real, neither is answered by a nicer approval card, and I used them as a reason to keep deliberating.

Then I sat down to write this post, which was going to be a tour of my careful narrow doors, and I went to check that they were narrow.

Three of them had holes

The database check authorizes an ID, not a file. CheckDatabase confirms you asked for a database you're allowed to use. Then db.execute opens that file and hands your SQL to SQLite unexamined. SQLite has a statement called ATTACH DATABASE, it's on by default, and nothing was turning it off. So one call did this:

ATTACH DATABASE '/path/to/savant-internal.db' AS ext;
CREATE TABLE stolen AS SELECT * FROM ext.secrets;

I wrote that as a test against the real code, and it read a row out of a database the script was never granted. It also walked past the file check on the way, because SQLite opens the attached file itself rather than going through the file API.

The domain allowlist was one redirect deep. fetch checked the first URL, then handed the request to Go's default HTTP client, which follows up to ten redirects without asking anyone. So the allowlist constrained the first hop and nothing after it. A 307 preserves the method and the body, which makes it an outbound pipe rather than a read. With an allowlist containing only localhost, my probe delivered method=POST body="secrets=exfiltrated" to a host that wasn't on the allowlist at all. (The probe hops between two loopback addresses, since that's what a test can reach. The mechanism doesn't care where the second host lives.) It doesn't need a malicious allowlisted domain. It needs one with an open redirect.

If you've read Simon Willison on the lethal trifecta, private data plus untrusted content plus a way out, that's all three in one script, and I'd shipped it.

The file check was a string comparison wearing a helmet. CheckFilePath ran filepath.Clean and compared the result against an allowed prefix. Clean is lexical. It collapses ../ and stops. There was no symlink resolution, and the unresolved path went straight to os.ReadFile, which follows symlinks the way the kernel always has. A script allowed to write in its own folder could put a symlink there and read whatever it pointed at.

The part I'd rather not print: Savant's filesystem tool server, a different module in the same codebase, has had symlink-attack protection the whole time. I knew the distinction. I'd already written the careful version somewhere else and never carried it across.

Three different doors. One bug.

Each check validated a name, then handed the real operation to something that resolved that name differently. CheckDatabase checked an ID and SQLite opened what the SQL said. CheckURL checked a hostname and net/http followed the chain. CheckFilePath checked a string and the kernel followed the link. The check and the use disagreed about what the argument meant, every time, and the gap between them was the hole.

This has a name, of course. It's the confused deputy, and it's older than I am. I rederived it the expensive way, in my own code, with a door metaphor.

The fix had the same bug in it

The fixes are unremarkable, which is the point. The database driver now installs an authorizer that denies SQLITE_ATTACH, rather than trying to spot bad SQL by reading it. fetch got its own HTTP client whose redirect policy re-runs the domain check on every hop, so a denied redirect is refused before the request goes out. And the file check resolves symlinks before it compares anything.

That last one is fiddly, because you have to resolve paths that don't exist yet. You're writing a new file, so there's nothing to resolve, but the folder above it might be a symlink and you still need to follow that. The code strips the final component, resolves the parent, then re-attaches. Applying .. to an already-resolved parent is what stops symlink/.. from escaping.

Then the review found that the fix leaked on relative paths.

The resolver called filepath.Abs on anything that wasn't absolute. filepath.Abs calls Clean. Clean collapses .. lexically, before any symlink gets resolved. So d/../secret.txt, where d is a symlink pointing out of the sandbox, cleaned to secret.txt, looked local, and passed. The kernel then followed d to somewhere else entirely and applied .. there. Check and use disagreed about what the path meant.

Read that again, because it's the same sentence I wrote about the original bug. The fix for a lexical-check-versus-real-resolution bug did a lexical collapse before real resolution. The resolution was to reject relative paths outright, since a sandboxed script handing you a path relative to the backend's working directory was never meaningful anyway.

I'd like to report that I spotted this myself. I didn't. It came out of review, after I'd already written the careful version.

Then, verifying the finished fix, I wrote a test that built its attack path with filepath.Join. Join cleans. It collapsed the .. out of my own exploit before the check ever saw it, the test passed, and I believed the wrong thing for the third time in one afternoon. That third one isn't the same bug, to be fair to me: it's a test that didn't test what I thought. Same lexical-versus-real distinction, third costume.

It is not a hard idea. Resolve the thing before you judge it. I knew it well enough to write a blog post about it and still got it wrong twice more while fixing it.

Then a reviewer asked what the redirect fix actually checks.

It re-runs the domain check on every hop. The domain check compares a hostname. DNS resolves hostnames, afterwards, to whatever they point at. So an allowlisted name whose record aims at your own machine never needed a redirect in the first place: it passed the check, and the check had no idea. I'd fixed the hop count and left the bug class sitting one layer down, in the same function, in the fix I'd just written for it.

That one's fixed now too, and the fix is the first thing in this whole sequence that actually applies the lesson instead of restating it. The address check moved into the dialer, in a Control hook that runs after resolution and inspects the address the connection is actually about to use. Not the name. Not a separately-resolved copy of the name, which would leave a window between the resolving and the dialing where the answer can change. The bytes going into the socket.

Check the thing you're about to do, at the moment you do it, in the code that does it. Four rounds to apply one sentence.

The other two didn't have holes. They don't need them.

If you build this kind of software you've been waiting to say it, so let me say it, and let me put it harder than a comment would.

Savant already runs tool servers. Those are third-party programs on your machine, in your account, that you installed because a README told you to. Nobody audited them. There's a shell tool server in the ecosystem. Users install it. And my own CheckMCPTool blocks exactly one thing, a recursion guard, and otherwise lets a script call any tool name that's in its manifest.

Which means a script can already run shell commands today, if you've installed that server and listed its tool. My own internal risk assessment says so, in a section I quoted while drafting the opposite argument: "Savant scripts can already invoke it via mcp.invoke if the user enables it."

The fifth door is worse, and I only found it because a reviewer went looking after the other four. CheckAgent validates an agent ID against the manifest, exactly like CheckDatabase validates a database ID. Then it runs the agent. An agent has its own tool selection, configured on the agent, not on the script. So agent.invoke gets the script whatever that agent can reach, through a language model, with variables the script may have built out of a web page it just fetched. It's the same check-a-name-then-hand-off shape as every bug above it, except nothing here is a bug. It's the design.

So: three doors had holes, and the two that didn't are wide open by construction. The door I spent three months refusing to build has been ajar the whole time, one indirection to the left, and the thing I called the permission that ends the list was never the frontier.

Here's where the objection stops working, though. "You already accepted worse risks" is the reasoning behind most bad security decisions, and it stays bad when I'm the one making it. A user who goes and installs a shell tool server crossed a barrier on purpose. Shipping cmd.run() puts the capability in every install and lets the model propose its own use of it, inside a card people are already clicking through. Same capability, different rate, different population, initiated by a different party. That's not nothing, and it isn't an argument for shipping.

What the user actually asked for

I should say this plainly, because it's the most obvious criticism and it's correct: the user's request has had an answer the entire time, and it's a pandoc.convert() function with a locked-down argument list. They wanted a PDF. Not a shell. It's an afternoon, it gives them exactly the thing they asked for, and it gives them none of the thing they didn't.

What's been open for nearly three months isn't their PDF. It's a general capability nobody requested, that I keep circling back to because general capabilities are more interesting to build than specific ones. Some of what I've been calling caution was appetite in caution's clothes, and the threat model was the respectable way to keep the question alive.

What I still don't know

The decision is genuinely open, and I'm not going to wrap this up with a verdict I don't have.

The one that decides it is: who am I defending against? If a determined attacker with prompt injection is in scope, nothing on my mitigation list is strong enough and the feature shouldn't exist. If it's honest users and AI mistakes, the stack is fine. That's not a technical question and I keep treating it like one.

Every lever that decides what a command may reach is an allowlist. None of them is a jail. Marcus Ranum was calling enumerating badness a losing strategy in 2005, and the interpreter problem, where blocking python and node and bash still leaves whatever I didn't think of on a Tuesday, only has no answer because I picked the weak mechanism. Allowing python is fine if python has no network and one file mounted read-only. The mechanisms exist: App Sandbox on macOS, a job object and a restricted token on Windows. Two platform files. I haven't priced the maintenance on either, so what I actually have isn't an impossibility, it's an unexamined cost, and I've been quoting the first when I meant the second.

And mcp.invoke and agent.invoke need a decision of their own, which the risk assessment doesn't account for, because I wrote that document as though command execution were hypothetical.

Where the wall is

The request is still open. Someone wants their Markdown to become a PDF, the tool is already sitting on their machine, and I've been writing threat models about a door I hadn't built while three of the five I had built stood open, and the other two were never doors.

I don't think the lesson is "ship it" or "don't." It's that I spent three months on the risk I could see coming, and shipped the same bug four times in the code I'd already written, twice of those while fixing it. It took writing a post about the frontier to make me look at the floor.

So tell me where the wall is. If you've shipped command execution in a product where an AI writes the commands, I want to know what you allowlisted and what you regretted. If you've decided not to, I want to know what your users did next. And if you've priced a per-subprocess jail on Mac and Windows and decided it was worth it, or wasn't, I'd like that most of all. Write to me at support@usesavant.com. The comments on a post like this are usually sharper than the post.

Every bypass described here is fixed, with regression tests, in Savant 2.9.58 and later. All were found by code review, not from any incident. I'm not claiming the whole class is sealed: the address guard covers the ranges that matter on a desktop machine, and I'd rather say that than "comprehensive." The mcp.invoke and agent.invoke reach described above is current behavior by design, and it's on the list.

Savant is a desktop AI workspace: one app for Claude, GPT, Gemini, Grok, and local models, with built-in tools for research, documents, images, email, and more. It's currently in an invite-only rollout. You can request access here.