AI 编程基准与风险库
直接答案：每个发现只能在其样本与限制范围内使用。可用性检查不会生成胜者结论，待执行的应用构建器记录仍然不是结果。
| 类别 | 工具 | 指标 | 证据类别 | 发现 | 限制 |
|---|---|---|---|---|---|
| coding-agent | Claude Code, Codex, Cursor, GitHub Copilot | task-result | reproducible-experiment | All four retained runs identified the root-cause module and produced a first-attempt patch that passed independent verification; Cursor did not run tests inside the restricted non-interactive mode. | One small fixture and one run per surface cannot establish general coding quality, speed, or reliability; Cursor's shell restriction is a mode boundary rather than a product-wide verdict. |
| code-review | Claude Code, GitHub Copilot | false-positive | reproducible-experiment | Claude Code mapped 8 of 9 seeded defects and one false-positive trap; GitHub Copilot CLI mapped 5 of 9 seeded defects and one false-positive trap. | A single synthetic diff and one run per tool do not establish general review recall or false-positive rates; the CLI surfaces are not the same as hosted pull-request bots. |
| app-builder | v0, Lovable, Bolt.new, Replit Ai | availability | reproducible-experiment | All four records remain pending, so no build, revision, export, deployment, handoff, or performance result is publishable. | This is an evidence-availability check, not evidence that any product is unavailable or failed the task. |
| coding-agent | Windsurf, Devin | availability | official-source | Windsurf pricing and getting-started URLs redirect to Devin pricing and Devin Desktop documentation. | A redirect establishes current public packaging, not the contract terms or migration status of every existing customer. |
| coding-agent | Cursor | security | official-source | Cursor documents a Privacy Mode that prevents training on customer code and can be enforced by teams. | This records a documented control, not an independent audit of implementation or a comparison with other products. |
方法与证据
限制
本地实验是小型 fixture 上的单次运行，不是分布。厂商页面不是独立审计。模型级基准不会映射为消费产品入口的结论。
来源与证据
来源
- Cognition source核验 2026-07-13高变化
Windsurf pricing and getting-started URLs redirect to Devin pricing and Devin Desktop documentation.
- Cursor source核验 2026-07-13高变化
Cursor documents a Privacy Mode that prevents training on customer code and can be enforced by teams.
证据
- 基准测试核验 2026-07-11
One small fixture and one run per surface cannot establish general coding quality, speed, or reliability; Cursor's shell restriction is a mode boundary rather than a product-wide verdict.
方法论 - 基准测试核验 2026-07-11
A single synthetic diff and one run per tool do not establish general review recall or false-positive rates; the CLI surfaces are not the same as hosted pull-request bots.
方法论 - 基准测试核验 2026-07-11
This is an evidence-availability check, not evidence that any product is unavailable or failed the task.
方法论 - 决策矩阵核验 2026-07-13
A redirect establishes current public packaging, not the contract terms or migration status of every existing customer.
方法论 - 决策矩阵核验 2026-07-13
This records a documented control, not an independent audit of implementation or a comparison with other products.
方法论