Home / How we rate

How we rate AI agents

Every tool gets a level from 1 to 5 on each criterion. Each level has a written rule, so two people applying it should land on the same score. The weighted total is the score out of 10.

What we look at

We read the vendor's documentation, pricing page, security and trust pages, changelog and terms. We check independent review platforms (G2, Capterra, Gartner Peer Insights, app marketplaces), public benchmark leaderboards and news coverage, and record each rating with the number of reviews and the date we saw it. Every claim in a review links to its source. Numbers that vendors publish about themselves, such as resolution rates or benchmark scores they ran, are shown as claims and never count towards a score.

We have not run these tools on our own workloads yet. Scores reflect what each vendor documents and what independent reviewers report; hands-on testing is planned.

Pricing, security and customer evidence follow the same rules in every category. "What it can do" and "Control" are written per category, because a coding agent and a support agent do different jobs. Where an independent tester runs hands-on tests across a category, as G2 now does for customer support agents, those results become their own criterion and carry the most weight.

Customer support

Independent tests · 30%

How did it do when an independent tester gave it real support tasks?

Level	Rule
5	A top-3 place on a leaderboard from a hands-on test G2 ran itself.
4	The top half of a G2-run leaderboard, or a score of 70 or more out of 100.
3	Tested by G2 with middling results: the middle of the leaderboard, a score of 60–69.
2	No hands-on test run by G2 (declined, not evaluated, or a partner benchmark only), or the bottom third of a G2-run test or a score below 60.
1	Scored below 40 out of 100 in a hands-on test G2 ran itself.

Only tests G2 runs itself count, because G2 asks readers not to compare its own results with partner benchmarks. When G2's views of the same agent disagree, we average the level each view gives and round down. This level is calculated from G2's published results, not set by hand.

What it can do · 20%

Can it resolve problems, not just answer questions, on the channels your customers use?

Level	Rule
5	Takes real actions in other systems on chat, email and voice, in 30+ languages.
4	Takes real actions in other systems; chat and email, voice missing or limited.
3	Answers plus limited actions, often only inside its own help desk; chat and email.
2	Answers from a knowledge base, on one main channel.
1	Scripted flows or very limited answers.

Pricing · 15%

Can you work out what you will pay before talking to sales?

Level	Rule
5	Full public price list including the AI unit price, with limits stated in numbers and no hidden minimum.
4	Public AI unit price, but it needs a paid base plan, seats or a stated minimum, or usage limits are vague.
3	Partly public: an entry tier is listed, larger tiers are by quote.
2	No public price; only indicative or third-party figures.
1	No price information anywhere.

Control · 15%

Can your team test it, set rules, hand off to people and see how it is doing?

Level	Rule
5	All five: testing before launch, guardrails, configurable handoff, quality analytics, versioning or approvals.
4	Four of: testing before launch, guardrails, configurable handoff, quality analytics, versioning or approvals.
3	Basic configuration, handoff to people and reports.
2	Minimal configuration.
1	A black box.

Security and data · 10%

Is your data handled to a standard you can show an auditor?

Level	Rule
5	SOC 2 Type II, ISO 27001 or 42001, HIPAA or data residency, and a clear no-training policy.
4	SOC 2 Type II plus one of: ISO 27001/42001, HIPAA or data residency, a clear no-training policy.
3	SOC 2 only, or an unclear policy on training with your data.
2	Security claims without named reports.
1	Nothing published.

Customer evidence · 10%

Do independent reviews and named customers back up the vendor's story?

Level	Rule
5	Rating of 4.5+ from 200+ reviews of this product itself, and named public customers. Ratings that cover only a parent platform count at most as level 4.
4	Rating of 4.3+ from 50+ reviews, or strong named customer stories with numbers.
3	Some independent reviews (fewer than 50) or a rating of 4.0–4.3.
2	Few or no independent reviews; vendor claims only.
1	Negative independent evidence: outages, security incidents, legal action, widespread complaints.

Level 1 applies when negative evidence outweighs the positive, for example when the review platform with the most reviews (100 or more) rates the product below 3.0. Security incidents, outages and lawsuits that do not outweigh the positive evidence are listed under concerns in each review.

Coding agents

What it can do · 30%

Can it take a task from plan to pull request on its own, locally and in the background?

Level	Rule
5	End to end on its own: plans, edits, runs tests, fixes failures and opens a pull request, locally and in the background, extensible with MCP or tools.
4	Autonomous multi-file edits that run commands and tests, but local only or cloud only.
3	Multi-file edits with step-by-step approval; limited command running.
2	Single-file edits or chat answers.
1	Autocomplete only.

Pricing · 20%

Can you work out what you will pay before talking to sales?

Level	Rule
5	Full public price list including the plan prices, with limits stated in numbers and no hidden minimum.
4	Public plan prices, but it needs a paid base plan, seats or a stated minimum, or usage limits are vague.
3	Partly public: an entry tier is listed, larger tiers are by quote.
2	No public price; only indicative or third-party figures.
1	No price information anywhere.

Control · 20%

Can a developer or a team decide what it may do, and undo it?

Level	Rule
5	All five: granular permission modes, sandboxed execution, diff review or PR-only workflow, checkpoints or undo, org policies and audit logs.
4	Four of those five.
3	Three of those five.
2	One or two of: permission modes, sandboxing, diff review or PR-only workflow, checkpoints, org policies and audit logs.
1	No controls documented.

Security and data · 15%

Is your data handled to a standard you can show an auditor?

Level	Rule
5	SOC 2 Type II, ISO 27001 or 42001, a zero-data-retention option, and a clear no-training policy.
4	SOC 2 Type II plus one of: ISO 27001/42001, a zero-data-retention option, a clear no-training policy.
3	SOC 2 only, or an unclear policy on training with your data.
2	Security claims without named reports.
1	Nothing published.

Customer evidence · 15%

Do independent reviews and named customers back up the vendor's story?

Level	Rule
5	Rating of 4.5+ from 200+ reviews of this product itself, or a top result on an independent benchmark, and named public customers. Ratings that cover only a parent platform count at most as level 4.
4	Rating of 4.3+ from 50+ reviews, or a published independent benchmark result, or strong named customer stories with numbers.
3	Some independent reviews (fewer than 50) or a rating of 4.0–4.3.
2	Few or no independent reviews; vendor claims only.
1	Negative independent evidence: outages, security incidents, legal action, widespread complaints.

Voice agents

What it can do · 30%

Can it hold a real phone conversation and get the caller's task done, on inbound and outbound calls?

Level	Rule
5	Everything in level 4, plus all three: 30+ languages, batch outbound campaigns, and the same agent on SMS or chat.
4	Calls your own APIs or tools during the call to act in other systems, on inbound and outbound calls, with transfer to a person.
3	Inbound and outbound calls with built-in actions only, such as booking a slot or transferring the call.
2	Answers questions by phone from a knowledge base; no actions in other systems.
1	Phone menus or a fixed call script.

Pricing · 20%

Can you work out what you will pay before talking to sales?

Level	Rule
5	Full public price list including the per-minute price, with limits stated in numbers and no hidden minimum.
4	Public per-minute price, but it needs a paid base plan, seats or a stated minimum, or usage limits are vague.
3	Partly public: an entry tier is listed, larger tiers are by quote.
2	No public price; only indicative or third-party figures.
1	No price information anywhere.

Control · 20%

Can your team test calls before launch, set rules and review every call?

Level	Rule
5	All five, plus team roles and audit logs.
4	Four of those five.
3	Three of those five.
2	One or two of: simulated test calls before launch, versioning or approvals, guardrails or redaction of personal data, configurable transfer rules, call review with transcripts and post-call analysis.
1	No controls documented.

Security and data · 15%

Is your data handled to a standard you can show an auditor?

Level	Rule
5	SOC 2 Type II, ISO 27001 or 42001, a HIPAA BAA or data residency, and a clear no-training policy.
4	SOC 2 Type II plus one of: ISO 27001/42001, a HIPAA BAA or data residency, a clear no-training policy.
3	SOC 2 only, or an unclear policy on training with your data.
2	Security claims without named reports.
1	Nothing published.

Customer evidence · 15%

Do independent reviews and named customers back up the vendor's story?

Level	Rule
5	Rating of 4.5+ from 200+ reviews of this product itself, and named public customers. Ratings that cover only a parent platform count at most as level 4.
4	Rating of 4.3+ from 50+ reviews, or strong named customer stories with numbers.
3	Some independent reviews (fewer than 50) or a rating of 4.0–4.3.
2	Few or no independent reviews; vendor claims only.
1	Negative independent evidence: outages, security incidents, legal action, widespread complaints.

App builders

What it can do · 30%

Can it go from a description to a working, published app, and fix its own mistakes?

Level	Rule
5	Everything in level 4, plus native iOS and Android apps it can send to the app stores, and connectors you can extend with MCP or your own APIs.
4	Full-stack web apps plus an agent that tests the running app and fixes its own errors, and built-in payments or connectors to outside services.
3	Full-stack web apps with a database, sign-in and one-click publishing.
2	Front-end code or single pages; the database, sign-in and hosting are left to you.
1	Mockups or templates only; no working app.

Pricing · 20%

Can you work out what you will pay before talking to sales?

Level	Rule
5	Full public price list including the plan prices, with limits stated in numbers and no hidden minimum.
4	Public plan prices, but it needs a paid base plan, seats or a stated minimum, or usage limits are vague.
3	Partly public: an entry tier is listed, larger tiers are by quote.
2	No public price; only indicative or third-party figures.
1	No price information anywhere.

Control · 20%

Do you own the code, and can you undo changes and keep a live app safe?

Level	Rule
5	All five.
4	Four of those five.
3	Three of those five.
2	One or two of: direct code editing, two-way GitHub sync or full code export, version history with restore, separate test and live environments, team roles with single sign-on.
1	No controls documented.

Security and data · 15%

Is your data handled to a standard you can show an auditor?

Level	Rule
5	SOC 2 Type II, ISO 27001 or 42001, a built-in security scan of the apps it builds, and a clear no-training policy.
4	SOC 2 Type II plus one of: ISO 27001/42001, a built-in security scan of the apps it builds, a clear no-training policy.
3	SOC 2 only, or an unclear policy on training with your data.
2	Security claims without named reports.
1	Nothing published.

Customer evidence · 15%

Do independent reviews and named customers back up the vendor's story?

Level	Rule
5	Rating of 4.5+ from 200+ reviews of this product itself, and named public customers. Ratings that cover only a parent platform count at most as level 4.
4	Rating of 4.3+ from 50+ reviews, or strong named customer stories with numbers.
3	Some independent reviews (fewer than 50) or a rating of 4.0–4.3.
2	Few or no independent reviews; vendor claims only.
1	Negative independent evidence: outages, security incidents, legal action, widespread complaints.

From levels to a score

Each level is divided by 5 and multiplied by the criterion's weight; the five results add up to 100 points, shown as a score out of 10 with one decimal. Ties are ordered by customer evidence, then by name.

Score	Label
9.0 – 10	Outstanding
8.0 – 8.9	Excellent
7.0 – 7.9	Very good
6.0 – 6.9	Good
5.0 – 5.9	Fair
below 5.0	Weak

Independence

No vendor pays for a rating, a position or a review. We do not use affiliate links. Vendors can send corrections with a public source, and we update the review and its date.