Your company’s codebase might be blocked from using third-party APIs, not out of paranoia, but because contracts require it. This could be the case for healthcare clients, defense contractors, or any employer with a strict IP policy. Still, most useful AI coding tools like Cursor, Windsurf, and Copilot Workspace automatically send your files, DOM structure, and business logic to external servers.
Cline works differently. Instead of connecting to Claude or GPT-4, you can use it with Ollama to turn it into a fully autonomous coding agent. It plans changes, edits files, and runs terminal commands, all on your own machine. This article covers a practical guide for setting up local agentic coding. It covers the context window issue that often confuses new users, explains which model sizes can handle multi-step agent loops, and points out where this setup still lags behind cloud agents, so you know what to expect.
The walkthrough later in this article uses a small Express API with duplicated request validation across three routes. You can find it in this GitHub repo. Clone the master branch to follow along with the refactor yourself; the solution branch has the finished Zod middleware if you want something to check your work against once you’re done.
Contract language, compliance rules, and company IP policies don’t bend for developer convenience. If your contract says code can’t leave the building, sending files to a cloud coding agent is a clear breach, even if it goes unnoticed. This setup is meant to give you an agent with the same planning and execution loop as Cursor or Copilot Workspace, but with all inference happening on your own hardware instead of someone else’s GPU cluster.
Cline doesn’t just add steps to autocomplete. Instead, it uses a two-mode loop that separates thinking from action.
In Plan mode, Cline reviews your codebase, asks follow-up questions, and suggests an approach before writing any code. This is your chance to catch any problems in the plan, such as scope changes, incorrect assumptions about your architecture, or ensuring the agent understands the task. No code is written yet.
In Act mode, Cline edits files, runs terminal commands, and, if the browser tool is enabled, uses Puppeteer to interact with the user interface and check its own actions. By default, you must approve every file edit and terminal command, so you don’t risk the agent making unwanted changes. There is an auto-approve option if you want to give it more control, but for your first local setup, it’s best to leave this off. This way, you can see what the agent plans to do before letting it proceed.
Two quick setup details to keep in mind:
Cline was first called Claude Dev, which is still reflected in its extension ID (saoudrizwan.claude-dev). Now, it’s independent of any single provider and supports over thirty LLM providers, including any endpoint compatible with OpenAI, like Ollama.
# macOS / Linux. See full instructions at https://ollama.com/download curl -fsSL https://ollama.com/install.sh | sh # pull a coding-capable model more on which one in the next section ollama pull qwen2.5-coder:32b
This will give you a model running locally at http://localhost:11434. Don’t set up Cline yet, because Ollama’s default context window can cause problems with agent tasks if you skip the next step.
Ollama’s default context window is 4,096 tokens, and sometimes drops to 2,048 if you have less VRAM. Either way, that’s not enough. Each Cline turn includes the system prompt, tool schemas, files read, and conversation history. If you go over the window size, Ollama won’t show an error. It just quietly drops the oldest tokens and keeps going. The agent doesn’t crash, but it might forget its plan, change files it already fixed, or create function signatures that don’t exist. If you skip this step, you could waste hours debugging what looks like a model problem, when it’s really a configuration issue.
Fix it at the model level with a custom Modelfile:
cat > Modelfile <<'EOF' FROM qwen2.5-coder:32b PARAMETER num_ctx 32768 EOF ollama create qwen2.5-coder-32b-agent -f Modelfile
32K is the minimum requirement, not the goal. Think of it as the lowest level where coding tools will work. If your VRAM allows, aim for 64K or 128K; otherwise, you might hit problems during a multi-file refactor. The actual cost of higher VRAM is discussed in the model selection section.
Open the Cline panel from the VS Code Activity Bar (the Cline icon on the far-left sidebar), then click the gear icon in the top-right corner of the panel to open its settings. If the sidebar’s hard to find, use the Command Palette (Cmd/Ctrl+Shift+P) and run Cline: Open Settings instead.
In Cline’s settings panel:
Ollamaqwen2.5-coder-32b-agent — that is the custom model which you have just created, not the base modelEven after fixing the num_ctx issue, the last field remains important. Cline needs to know the window size to manage its context budget correctly. If Ollama and Cline have different window sizes, you’ll run into the same silent-truncation problem you just fixed.
At this point, Cline talks to your local model exactly as it would to Claude or GPT-4: the same Plan/Act loop, the same approval prompts, and the same file edits. The only difference is where the inference happens.
If Cline throws Error: the operation timed out on a task that should’ve worked, this is probably why: Cline’s Ollama provider has an open bug (cline#6549) where the connection drops if the model takes longer than 5 minutes to produce its first token, no matter what timeout you configure. This shows up most on larger models with a slow prefill, big prompts with a lot of file context loaded in, or hardware that’s spilling the model out of VRAM onto CPU.
Before assuming your config is wrong, rule out the boring causes first: confirm ollama serve is actually running (curl http://localhost:11434/api/generate -d '{"model": "your-model", "prompt": "hi", "stream": false}' should respond, not hang), and preload the model with a throwaway prompt so Cline’s real request isn’t also waiting on a cold load. If it’s genuinely the 5-minute limit and not one of those, the workaround is to switch API Provider to OpenAI Compatible instead of Ollama, with the Base URL set to http://localhost:11434/v1. That routes through a different code path in Cline that isn’t subject to the same hardcoded limit.
Not all local coding models can run an agent loop. The workflow Cline uses, reading files, making a plan, calling tools with structured arguments, interpreting results, making revisions, and repeating, requires a series of decisions where mistakes can build up. Smaller models often struggle here. For example, a 7B model might call a tool with the wrong arguments, forget a constraint it agreed to earlier, or say a task is done when it isn’t. This isn’t about lacking knowledge, but about following instructions over a long context. That’s why both the number of parameters and training for tool use matter more here than with autocomplete.
# The current sweet spot for a 24GB+ GPU MoE architecture, # 256K context, trained specifically for agentic tool-calling ollama pull qwen3-coder:30b # The one local coder with a published agentic benchmark. # Achieved 46.8% on SWE-Bench Verified with just 14GB of memory. ollama pull devstral:24b # Strong on raw code-repair, not agentic-tuned, capped at # 32K context pair with the Modelfile fix from the setup section ollama pull qwen2.5-coder:32b
You need at least 24GB of VRAM for a smooth experience. With less, you’ll trade capability for a lower hardware budget, which means more retries, more corrected plans, and more hands-on supervision.
If you’re working with limited VRAM, don’t reduce a 30B-plus model to a quantization level that severely damages its performance. Instead, go with qwen3.6:35b-a3b; this is a mixture-of-experts model that activates only a fraction of its parameters for each token, meaning that even though the model is larger in total size, it places less demand on active computing and still comes with native tool-calling. This represents a better compromise than running a dense 32B model at 16GB and watching it get swapped to disk during the task.
These models don’t fully match the top cloud models, but they are good enough for real agent work. The remaining gaps and where they matter are covered in the limits section below.
Here’s an example of the kind of task this setup is designed for: a small, well-defined refactor using qwen3-coder:30b on hardware with at least 24GB of VRAM, as recommended earlier. If you try it yourself, your exact results, like the plan wording, edit order, or test timing, may differ, but the overall workflow stays the same: Plan, then Act, then verify.
The requirement is to consolidate the request validation currently duplicated across three route handlers. Those for POST /users, POST /orders, and PATCH /users/:id. Each one implements its own if checks, and to refactor it into a common middleware using Zod, without in any way altering the actual validation behavior of the routes.
Prompt Cline directly in the chat panel:
The validation logic in routes/users.js and routes/orders.js is duplicated. Extract it into a shared middleware using Zod, apply it to all three routes, and don’t change what each route currently accepts or rejects.
To confirm that Zod is already a dependency, Cline looks at all three route files along with package.json, then sets out its plan: to define a schema for each route, to create a generic validate(schema) middleware factory, and to specify where it should be incorporated. At this stage, no files have been altered; this is all part of the reasoning phase.
Review this plan before approving it. This is your chance to catch any hidden validation rules in the route handlers. If you miss them, you could end up with silent changes in behavior in other files.
Switch to Act mode, and Cline executes. The first step is for it to create the middleware:
Learn more →// middleware/validate.js
import { z } from 'zod';
// This turns a Zod schema into Express middleware.
// Returns an error of 400 due to field-level errors rather than giving a general 500 error.
export function validate(schema) {
return (req, res, next) => {
const result = schema.safeParse(req.body);
if (!result.success) {
return res.status(400).json({ errors: result.error.flatten() });
}
req.body = result.data; // parsed and changed to correct types, not just checked
next();
};
}
Each file edit pauses for your approval before it’s written. Next, it updates routes/users.js, routes/orders.js, and again routes/users.js for the PATCH handler, replacing each inline if block with a call to the schema and validate(). A 7B model might lose track of which route needs which fields, but qwen3-coder:30b can keep this straight because the schemas were fully defined during the Plan phase, not created on the fly during editing.
Cline runs the test suite as a terminal command, which also requires approval:
npm test -- routes
Each of the three routes has the same accept/reject behavior as before the refactor, and no tokens were sent anywhere outside localhost:11434.
The PATCH /users/:id schema is the part most likely to cause problems during the refactor, regardless of model size. It’s easy to copy a schema from POST /users, where every field is required, and apply it directly to the PATCH route, where every field should be optional. Pay close attention to this when reviewing the changes: if the new schema doesn’t mark fields as optional (or isn’t wrapped in something like .partial()), the PATCH route will start rejecting valid partial updates it used to accept. This is exactly the kind of silent behavior change the test suite is meant to catch. A stronger model is more likely to spot and fix the mismatch during the task, while a weaker one might apply the schema as written and rely on the tests to catch the mistake. That’s still fine, as long as you actually run the tests and don’t just trust a green “done” from the agent.
Claude Sonnet 5 achieves an 85.2% score on SWE-bench Verified, while the best fully local coding model in this scenario, devstral:24b, manages only 46.8%. The 480B version of qwen3-coder, which you very likely cannot run on consumer-grade hardware, attains 27.2% on SWE-bench when using a SWE-Agent harness, a result that is worse than that of the 24B model you actually can run, which shows that benchmark figures don’t always translate smoothly across different harnesses and should therefore not be taken at face value. Yet even if you interpret this generously, you’re still looking at about half the task-completion rate of a leading cloud-based model when it comes to the type of real-world GitHub-issue tasks that Cline is designed to automate.
The gap shows up in how the models behave, not just in the numbers. Cloud agents usually recover from their own errors more often and need fewer correction rounds. Local models, even the best ones, need more supervision, more rejections and retries, and require you to define the task scope more clearly. The PATCH-schema mistake was easy to fix, but that’s not always the case. If a task is more ambiguous or involves more files, a local model is more likely to make a mistake and stop rather than catch the error.
Don’t give a local setup an open-ended request like “improve error handling across the API.” Instead, treat it like assigning a task to a junior engineer: define one clear file boundary, one specific behavior to keep, and one way to check that it worked. This isn’t a permanent limitation of local models; it’s just how you get reliable results from smaller models right now.
Cloud agents are charged per token, while local agents require GPU resources. The qwen3-coder:30b model and the other models mentioned need 24GB of VRAM to run comfortably; that’s an RTX 4090 or 5090, or a Mac with 32GB or more of unified memory. Such hardware costs between $1,500 and $2,500, assuming you don’t already have it. In contrast, Sonnet 5 costs $2 or $10 per million tokens; even with a heavy day of Cline usage, tokens consumed stay in the low tens of millions, keeping total cost well below $50. The hardware only pays for itself compared to cloud expenses if you run agent workloads every day for months.
Cloud pricing also hides another cost: while Cline is running, your GPU is fully dedicated to inference. If you try to run another GPU-heavy process, like a build, a local dev server with GPU acceleration, or another model, you’ll notice the impact.
This isn’t an argument against using local agents. It’s a reason to use them when the limitation is the main concern, like when you have proprietary code that can’t legally leave your machine, you’re working in an air-gapped environment, or your client contract doesn’t allow sending code to an external API, no matter how good the results are. In these cases, a 46% SWE-bench score running on localhost is better than an 85% score you can’t use. Make sure you understand your situation before investing in hardware.
Running local agentic coding is ultimately about control, not raw performance. Once you lock in your context window and pick a model designed for tool use, you get a solid, private environment that never leaks your proprietary codebase to a third-party API.
Use this setup when the constraint is the point: air-gapped environments, strict IP policies, or non-negotiable client contracts, not because it beats a cloud agent on capability. It doesn’t. But when privacy is a hard requirement, a solid local workflow easily beats an 85% SWE-bench score that you aren’t legally allowed to use.
Compare TanStack Charts, Recharts, and Chart.js by building the same React dashboard three times. See how each library’s mental model affects your code.
Is the Rust React Compiler’s 10x speed claim real? We put Babel, Vite (Oxc), and Bun to the test on a real app to find out where the speed actually matters.
Compare the top AI development tools and models of September 2026. View updated rankings, feature breakdowns, and find the best fit for you.
Which Markdown library is right for your React docs site? We rebuilt the same documentation app using TanStack Markdown and react-markdown to compare bundle size, setup complexity, and performance.
Hey there, want to help make our blog better?
Join LogRocket’s Content Advisory Board. You’ll help inform the type of content we create and get access to exclusive meetups, social accreditation, and swag.
Sign up now
