OpenCodeReview AI Code Review hands-on: how many bugs, security issues, and quality findings does it catch on a real Git diff?

First: what OpenCodeReview actually reviews

People often mash this together with OpenCode (the coding agent behind opencode serve). Different product. OpenCodeReview is Alibaba’s internal AI review assistant, open-sourced as a CLI: it reads Git changes, emits structured line comments, ships multi-language rules (NPE, thread safety, XSS, SQL injection), and talks to OpenAI- and Anthropic-compatible endpoints. Source: alibaba/open-code-review. Docs: open-codereview.ai.

It is not “another bot that compliments your PR”. A chat model only sees the snippet you pasted. It does not open callers, and it does not know whether the next file already holds the lock. OpenCodeReview’s bet is deterministic triage plus a deep agent: decide what to review, inject a file-type checklist, then let the agent file_read, search the tree, and compare other changed files.

We already published how to size 4GB / 8GB / 16GB for self-hosted OpenCode — that is the coding agent’s RAM bill. This article is only ocr review: automatic Git-diff, bug, security, and quality checks, and how many issues a real project actually yields.

The vendor line is “tens of thousands of developers, millions of defects”. That is Alibaba-scale history, not a promise for your next PR. If you want a second pair of eyes before merge, it fits. If you want a machine to sign off production, neither the README nor this hands-on will sign.

How it reads a Git diff: three modes

If you searched “auto-check Git diff”, you care about “what I changed”. The CLI splits that into three mutually exclusive modes — mixing them is a hard error. Flags: CLI Reference.

Range mode uses merge-base, which is closer to how humans review a PR than a raw two-branch diff: later commits that landed on main do not flood the review. The official GitHub Action does the same: compute merge-base, run ocr review --from MERGE_BASE --to HEAD_SHA --format json, then post inline comments.

A fourth path is ocr scan: whole files, no meaningful Git history required — useful when you inherit a directory. Tokens and noise go up. For merge review, start with a diff; scan the legacy tree later.

Always run ocr review --preview first. It walks the filter pipeline and spends no model money. You see will_review versus dropped-for-extension / deletion / test-path. In our run, a PR with images and lockfiles lost about half the “looks huge” files before the LLM ever saw them — correctly.

Five minutes: install, model, preview

npm global is the default install. There is also install.sh and a GitHub Release static binary. You need Git ≥ 2.41. Without a configured model, ocr review will not invent comments (unless you use Delegation Mode and hand the review to a coding agent you already pay for).

npm install -g @alibaba-group/open-code-review
ocr version
ocr config provider          # built-in or custom endpoint + key
ocr config model
ocr llm test                 # if this fails, stop burning tokens

cd /path/to/your-repo
ocr review --preview         # free: which files will be reviewed
ocr review --from main --to HEAD --format json --audience agent

Config lives in ~/.opencodereview/config.json. In CI, ocr config set writes provider / model / api_key, or reuse ANTHROPIC_* / OCR_LLM_*. Comment language is ocr config set language English (English is the default). Commit .opencodereview/rule.json for the repo; keep keys out of git.

The npm wrapper auto-updates in the background (~18 minute cooldown). Pin the version in CI and export OCR_NO_UPDATE=1 so Monday’s rules match Friday’s. Sessions sit under ~/.opencodereview/sessions/. ocr viewer defaults to localhost:5483 for replay, not a public service.

Do not copy the OpenCode self-host RAM article for this binary. The heavy work is the LLM API and concurrent git subprocesses; the CLI does not keep an LSP warm. Default --concurrency is 8. A laptop lid-close mid-range review leaves a session you must --resume. A machine that stays awake is more honest than gambling one ocr review on a travel notebook.

Hands-on: what three real change packs found

Same config (OpenAI-compatible protocol, mid-size chat model, system rules merged with a project rule) against three PR-shaped change packs — not a ten-line toy repo. Each pack: --preview, then ocr review --from --to --format json. We labelled comments: must-fix, should-fix, style/nit, false positive. One snapshot; swap the model and the counts drift. The mix matters more than the headline total.

Must-fix (7): three security, four defects. Security: string-built queries, unescaped output into HTML, internal error text returned to the client. Defects: nil used as an object, concurrent map write without a lock, errors not wrapped with %w so callers lose the cause. Those land in the advertised ruleset — NPE, thread safety, XSS, SQL injection — not a chat essay.

Should-fix (6): missing timeouts, retries on non-idempotent calls, logs that included an email or a request body. A human should raise them; Friday afternoon often does not. The agent does not get tired.

Style/nit (10) is the largest bucket. Names, early returns, “extract this function”. Useful, and it dilutes the signal. Leave it untuned and by week three the team folds the whole review — the same pathology people report on noisier PR bots: high recall, collapsing trust.

False positives (4) were boring in a good way: fixture SQL treated as production injection; an already-parameterized query scolded again; a pre-Go-1.22 loop-variable note on new code. Three of the four die if you exclude test paths in project rules after --preview. Blame the file filter before you blame the model.

That scale matches the official example: a range review often looks like “9 files, a dozen comments, about a minute, ~20k tokens”. Our three packs are roughly three isolated ranges, not one mega-PR. A single-file diff that eats ~80% of the token window is dropped before the LLM — generated blobs and lockfiles should vanish in preview, not on the invoice.

Bugs, security, quality: what it catches and misses

Split the 27 comments by the four words in the title. That is closer to how you will use the tool than a single “issues found” number.

Because the agent can read the whole file, it beats “green lines only”. The official map-without-lock example is the type: the diff adds one assignment; the shared state already lived above. Diff-only bots miss it; file_read pulls the context in. That is the claimed gap versus PR reviewers that only see the patch — those are faster and quieter, and weaker across files.

We also planted misses on purpose: a wrong discount formula, a currency field treated as USD, an endpoint that should stay closed before approval. Zero comments. The model does not have your domain glossary. --background with the PR description helps. It still does not replace a human on requirements.

Treat security comments as leads, not verdicts. “This concatenates SQL” still needs you to check the driver, the role, and the data class. This article will not show how to exploit those lines — only whether the reviewer marked them. If you also need a write-boundary for coding agents, see how agents edit files safely with a VFS — review is read-only; the coding agent writes disk.

CI: GitHub Actions, rules, false positives

After local works, teams want inline comments on the PR. The official composite Action checks out a trusted base (it does not materialize an untrusted fork worktree), fetches head git objects, computes merge-base, runs ocr review --format json, and posts line comments plus a sticky summary. Triggers: PR opened/synchronized, or a /open-code-review comment.

Least privilege: contents: read; add pull-requests: write only if you post reviews. Keys in Secrets, model names in Variables. Forks do not get repo secrets by default — that is protection, not a bug. Do not pair pull_request_target with a checkout of untrusted code just to review forks; that combination leaks secrets. GitHub’s own note: Security hardening for GitHub Actions.

Rules are four layers: --rule, project .opencodereview/rule.json, home-dir global, built-in system. ocr rules check path/to/File.java shows which layer won. Set merge_system_rule: true so two custom log notes do not wipe the Java NPE checklist. Excluding tests, generated code, and vendor is cheaper than arguing about false positives later.

On macOS, reviewing Xcode diffs is easy; keeping the runner alive is not. Setup and acceptance: GitHub Actions macOS self-hosted runner. A machine that does not sleep, with keys that do not share a laptop’s quota, is what makes review repeatable.