Choosing agent tools: a decision framework for editors, terminals, frameworks and automation
Which kind of agent tool fits my work, and how do I compare candidates fairly?
Start with where the work happens. An agent inside your editor supports a different way of working from a scheduled assistant or a framework embedded in your own product. Choosing the model is only one part of choosing the tool: the same model behaves differently depending on how the tool assembles context, which actions it permits and how it checks results.
This guide follows the agent fundamentals series. The main text is a product-neutral framework; named products, their licenses and surfaces are in Current options. It describes documented design choices, not a hands-on speed ranking.
Key points
- Classify the tool first: interactive (editor, terminal), delegated (cloud worker that returns a pull request), persistent (personal assistant on a schedule), embedded (SDK or framework in your product), or orchestrated (workflow automation).
- Ask three openness questions separately: can I inspect and change the tool’s code, can I run the model myself, and who operates the service that receives my data?
- The decisive differences are often not the model: context assembly, permission model, execution location, review surface and recovery from failure.
- Compare candidates on your own small tasks with a fixed protocol, and score cost per accepted result, not price per call.
- Check exit paths before committing: export of outputs and history, reproducible setup, and what happens when the service is unavailable.
Five kinds of agent tool
| Kind | Where it runs and how you interact | Best when | Main risk to manage |
|---|---|---|---|
| Interactive editor or terminal agent | On your machine or a dev box; you watch and steer each step | You want to stay in the loop and review diffs as they appear | Approving actions without reading them |
| Delegated cloud worker | In a provider’s environment; returns a branch or pull request | Well-scoped issues with good tests | Changes that pass tests but miss intent |
| Persistent personal assistant | A long-running service connected to chat channels and schedules | Recurring personal tasks across apps | Broad credentials; injected instructions in content it reads |
| Embedded SDK or framework | Inside your own application | The agent is part of your product | You own state, evaluation and operations |
| Workflow automation platform | A visual or declarative flow with model steps | Mostly fixed steps with a few judgment calls | Silent failures in error paths |
A non-programmer should usually start with an existing application’s supported interface and a reversible task. A developer should start with one tool and one verifiable result before adding multiple agents. A team should include account administration, audit logs and recovery in any trial rather than evaluating only the best demonstration.
Separate the tool, the model and the service
An open-source client can call a closed hosted model. An open-weight model can be served through a paid API. A downloadable application can still send prompts to a remote service. Ask three separate questions:
- Code: can I read, modify and redistribute the tool? Check the repository’s license file, not the marketing page. Some projects use a standard open-source license; others use “source-available” or “fair-code” licenses with commercial restrictions; some keep part of the code under a separate enterprise license. [1, 12]
- Model: which models can the tool use, can I bring my own, and can any of them run locally?
- Service: which infrastructure receives my files and prompts, what is retained, and under what terms?
Choosing models explains these licensing layers in detail, including weight licenses and the difference between open-source and open-weight.
What actually differs between tools
When two tools use the same model, their results still differ because of the machinery around it:
| Dimension | Question to ask | Why it matters |
|---|---|---|
| Context assembly | How does it find relevant files, docs and history? Does it read project instruction files? | Most wrong edits come from missing or wrong context |
| Tool and permission model | Which actions need approval? Can I allow read-only tools and gate writes? | Determines what an error can damage |
| Execution location | Local shell, container, remote VM, or provider cloud? | Determines data exposure and reproducibility |
| Review surface | Can I see each diff, command and source before accepting? | You can only catch errors you can see |
| Extensibility | Does it support skills, MCP servers or custom tools? | Lets you add your procedures instead of re-explaining them |
| Recovery | Can it resume after a crash, undo a change, or continue in a new session? | Long tasks fail partway; recovery decides the cost |
| Observability and audit | Are actions logged in a form a team can review? | Required for shared or regulated work |
The first and fourth rows are usually the most important for code: a tool that shows you a clear diff and runs your tests is safer than a slightly stronger model behind an opaque interface.
Run a small, fair comparison
Use three to five tasks from your real work: an ordinary success case, an ambiguous request that should trigger a clarifying question, and a task with a deliberate tool failure (a missing dependency, a failing test). For coding, include an existing regression test and a case that catches an incomplete fix. Run each candidate more than once, because model output varies.
Keep these fixed across candidates: the starting revision, the task text, the test command, the allowed actions and the budget. Record for each attempt:
- exact application and model version, and enabled tools;
- pass or fail against an acceptance check written before the trial;
- unrelated changes and whether recovery from the failure worked;
- elapsed time, human interventions and corrections;
- spend, including subscriptions or infrastructure where they apply.
Then compare cost per accepted result: total spend divided by the number of outputs that passed. A cheaper tool that needs three attempts and a manual fix can cost more than an expensive one that passes first time. The model-evaluation guide explains how to read pass rates and their uncertainty on small samples.
Do not install every available connector for the trial. Add the ones the task needs and verify the agent can use them; a long tool list creates selection ambiguity and makes failures harder to trace.
Before you commit
- Exit: can you export outputs, history and configuration in a usable format?
- Reproducibility: can a colleague recreate the environment from files in the repository?
- Routing: do you know which provider and model handle each request, and can you pin a version?
- Continuity: what happens when the service or the model you depend on is unavailable or retired?
- Self-hosting cost: include upgrades, dependencies, GPU capacity and incident handling, not just the license.
Choose the candidate that completes your representative tasks with the fewest unacceptable failures and manageable supervision. Keep the trial tasks: they become your regression suite when the next release arrives. Then design the workflow around the tool.
Current options, checked 6 November 2026
The tables below are representative examples, not a complete market survey or a ranking. Licenses were read from each repository’s license file on the check date; hosted-service terms are separate and can change.
Coding agents
| Tool | Kind | Code license or status | A useful first trial |
|---|---|---|---|
| Codex CLI | Terminal agent | Apache-2.0 client; model access and service terms separate [2] | A small repository repair with an executable test |
| Claude Code | Agent in the terminal, IDE extensions, a desktop app and the web | Proprietary product [3] | A multi-file change guided by repository instructions |
| Cursor | AI code editor with an agent mode | Proprietary application and service [4] | Edit, inspect a diff and revise one feature |
| GitHub Copilot | On GitHub.com: repository chat, a background agent that opens pull requests, and code review | Hosted product, account-specific access [5] | A bounded issue whose artifact is a pull request |
| Gemini CLI | Terminal agent | Apache-2.0 client; provider access separate [6] | A task that reads local files and uses an approved tool |
| OpenHands | Agent platform; local by default, optional cloud | MIT-licensed repository [7] | A task in an isolated container workspace |
| Aider | Terminal pair programmer | Apache-2.0 [8] | A narrow change where you review each patch |
| OpenCode | Terminal agent with many providers | MIT [9] | The same task with two different model setups |
| Cline | Editor extension, CLI and SDK | Apache-2.0 [10] | A change where each tool action is approved visibly |
Frameworks and automation
| Tool | What it supplies | Code license | What you still design |
|---|---|---|---|
| LangGraph | Stateful graph orchestration with persistence | MIT [11] | State schema, replay behavior, task checks |
| OpenAI Agents SDK | Agents, tools, handoffs, tracing | MIT [13] | Business logic, credential boundaries, acceptance tests |
| CrewAI | Multi-agent task orchestration | MIT [14] | A division of work that actually helps |
| Browser Use | Browser-automation agent library (plus a hosted offering) | MIT [15] | Authentication handling and post-action checks |
| n8n | Visual workflow automation with AI steps | Sustainable Use License (“fair-code”); enterprise files under a separate license [12] | Triggers, error paths, data contracts, delivery |
| OpenClaw | Persistent personal-agent gateway: channels, sessions, schedules | MIT [16] | Procedures, execution policy, operating routine (part 3) |
A browser-automation library is not a reason to drive a web page for an operation that a reliable API already provides; prefer the API and use the browser for what only the interface exposes.
Further reading
- Anthropic, Building effective agents: when you need an agent at all, and when a workflow suffices. [17]
- Anthropic, Writing effective tools for AI agents: what makes a tool easy for a model to use correctly, useful when judging a tool’s built-in actions. [18]
- Open Source Initiative, The Open Source Definition: the reference for what “open source” means for code licenses. [1]
- Each tool’s own license file (linked in the tables): the only reliable answer to “is it open?”.
References
Official repositories and documentation, accessed 6 November 2026:
- Open Source Initiative, “The Open Source Definition.” Definition.
- OpenAI, “Codex CLI.” Repository and license.
- Anthropic, “Claude Code overview.” Documentation.
- Cursor, product documentation. Documentation.
- GitHub, “Copilot on GitHub.com and GitHub Mobile.” Documentation.
- Google, “Gemini CLI.” Repository and license.
- OpenHands contributors, “OpenHands.” Repository and license.
- Aider contributors, “Aider.” Repository and license.
- Anomaly contributors, “OpenCode.” Repository and license.
- Cline contributors, “Cline.” Repository and license.
- LangChain, “LangGraph.” Repository and license.
- n8n, “n8n” and its license file. Repository, license.
- OpenAI, “OpenAI Agents SDK.” Repository and license.
- CrewAI, “CrewAI.” Repository and license.
- Browser Use contributors, “Browser Use.” Repository and license.
- OpenClaw contributors, “OpenClaw.” Repository and license.
- Anthropic, “Building effective agents,” 19 December 2024. Article.
- Ken Aizawa, “Writing effective tools for AI agents,” Anthropic, 11 September 2025. Article.