Semgrep vs. CodeQL vs. OpenTaint: XSS Detection Depth Compared

We tested Semgrep, CodeQL, and OpenTaint on five progressively harder XSS cases in a Java Spring application — from direct returns to builder patterns with virtual dispatch — to show where each tool's analysis model hits its limit.

Mar 24, 2026

Good rules are a big part of what makes a SAST tool accurate, and that isn’t going to change. What has changed is how easy rules are to write. Encoding a known vulnerability pattern as a rule used to take real expertise. Now AI can handle most of that work, and a friendlier rule format lets it handle more of the rest. So rules themselves aren’t really where tools differ anymore. The harder problem — the one no amount of rule tuning can fix — is the engine itself: how far it can actually trace a value through the code. If the engine can’t follow data through a constructor or a virtual call, even a perfect rule won’t catch the bug.

To see where that limit falls, we tested three tools — Semgrep, CodeQL, and OpenTaint — on five XSS examples in a Java Spring application. Every example is the same basic bug: a controller reads a request parameter and writes it straight into the HTML it returns. What changes is how the value gets from input to output. The first case returns it directly. After that it passes through a local variable, then a helper method, then a constructor chain, and finally a builder that uses virtual dispatch. Each step adds more code between the user input and where it’s used, and makes the bug a little harder to trace.

Each case measures two outcomes: false negatives (vulnerabilities the tool fails to detect) and false positives (secure code paths the tool incorrectly flags). Every case after the first pairs the vulnerable endpoint with a sanitized variant the tool should leave alone. The three tools under test:

Semgrep matches patterns syntactically and offers a taint mode for local dataflow. Its paid commercial edition, Semgrep Code, adds broader inter-procedural coverage. Results below distinguish Semgrep CE and Semgrep Code where they diverge.
CodeQL runs semantic analysis through a dedicated query language. We use its default java/xss rule. Free for open-source repositories. Private repos require GitHub Advanced Security.
OpenTaint interprets Semgrep-style patterns as dataflow queries — metavariables are tracked as program values, not syntactic placeholders. Runs whole-program analysis against a build artifact, which is what enables the deeper tracking shown in the later cases. Java and Kotlin today, Apache 2.0 / MIT licensed.

Five test cases

All five cases are the same bug: user input reaching an HTML response without escaping. What changes from one to the next is how far that input travels and how much code a tool has to trace to follow it.

#	Capability required	What the code does
1	Syntax matching	User input returned directly
2	Local dataflow	Value assigned to variable before return
3	Inter-procedural analysis	Value flows through a helper method
4	Field sensitivity	Value passes through constructor chains and nested objects
5	Pointer analysis	Value flows through builder pattern with virtual dispatch

These are ordinary patterns — a variable, a helper method, a constructor, a builder. Each one adds a step where a tool can lose the thread between the input and the sink.

Syntax matching — direct return

Here a profile page takes a message from the URL and writes it back into an HTML response. This is the simplest case: one endpoint, one parameter, no helpers. The controller below implements it.

@GetMapping("/profile/display")
@ResponseBody
public String displayUserProfile(
        @RequestParam(defaultValue = "Welcome") String message) {
    return "<html><body><h1>Profile Message: " + message + "</h1></body></html>";
}

A Semgrep rule to detect this pattern:

id: pattern.xss
patterns:
  - pattern: |
      @$ANNOTATION(...)
      $RETURNTYPE $METHOD(..., $TYPE $PARAM,...) {
        ...
        return "..." + $PARAM + "...";
        ...
      }

Results:

✅ Semgrep, ✅ CodeQL, ✅ OpenTaint: All three detect the vulnerability — no surprise for the simplest form of XSS.

Local dataflow — variable assignment

The same endpoint, but with the input assigned to a local variable before the return. That single step breaks AST-pattern matchers: the tool now needs local dataflow tracking.

@GetMapping("/profile/status")
@ResponseBody
public String displayUserStatus(
        @RequestParam(defaultValue = "Active") String message) {
    String statusMessage = "<html><body><h1>User Status: " + message + "</h1></body></html>";
    return statusMessage;
}

The vulnerable value is first assigned to statusMessage and then returned. The pattern rule from the first case no longer matches because the return statement contains a variable, not a concatenation.

The OpenTaint rule for this case is simpler than the first — because the engine treats the pattern as a dataflow query, not a syntax match:

id: pattern.xss
patterns:
  - pattern: |
      @$ANNOTATION(...)
      $RETURNTYPE $METHOD(..., $TYPE $PARAM,...) {
        ...
        return $PARAM;
        ...
      }

This rule captures the core condition: user input reaches a return statement. OpenTaint tracks $PARAM through the assignment and finds it in statusMessage.

Semgrep can handle this scenario too, but requires switching to a taint rule with explicit source and sink definitions:

id: taint.xss
mode: taint
pattern-sources:
  - patterns:
      - focus-metavariable: $PARAM
      - pattern: |
          @$ANNOTATION(...)
          $RETURNTYPE $METHOD(..., $TYPE $PARAM,...) {
            ...
          }
pattern-sinks:
  - patterns:
      - focus-metavariable: $SINK
      - pattern: |
          @$ANNOTATION(...)
          $RETURNTYPE $METHOD(...) {
            ...
            return $SINK;
            ...
          }

Both approaches work. The difference is how much of the dataflow model has to be expressed in the rule itself. In OpenTaint, the engine infers the flow. In Semgrep, the rule author declares it.

Handling sanitization

The secure version escapes user input before including it in the response:

@GetMapping("/profile/secureStatus")
@ResponseBody
public String displaySecureUserStatus(
        @RequestParam(defaultValue = "Active") String message) {
    String htmlContent = "<html><body><h1>User Status: " +
            HtmlUtils.htmlEscape(message) + "</h1></body></html>";
    return htmlContent;
}

The basic rules above still flag this secure version as vulnerable because they don’t recognize the sanitization function. For the pattern rule, the fix is a negative pattern. This only matters for OpenTaint — Semgrep’s pattern rule already misses this case entirely, so there is nothing for a negative pattern to suppress:

# pattern.xss — with sanitization
patterns:
  - pattern: |
      @$ANNOTATION(...)
      $RETURNTYPE $METHOD(..., $TYPE $PARAM,...) {
        ...
        return $PARAM;
        ...
      }
  - pattern-not: |
      @$ANNOTATION(...)
      $RETURNTYPE $METHOD(..., $TYPE $PARAM,...) {
        ...
        return HtmlUtils.htmlEscape($PARAM);
        ...
      }

Taint rules use sanitizer patterns instead:

# taint.xss — sanitizer
pattern-sanitizers:
  - pattern: HtmlUtils.htmlEscape(...)

Results:

❌ Semgrep (pattern): Misses the vulnerability — AST-pattern matchers cannot follow the assignment.
✅ Semgrep (taint): Detects the vulnerability and can recognize sanitization.
✅ CodeQL and ✅ OpenTaint (pattern and taint): Correctly handle both vulnerable and secure code.

From this point forward, Semgrep’s taint rules are used — pattern rules are insufficient. OpenTaint’s pattern rule from this case is reused unchanged for all remaining examples. Results are shown for both rule types.

Inter-procedural analysis — function call boundary

Now move the concatenation into a private helper. The controller looks clean, but the dangerous string is built inside the helper, one call deeper. To catch it, the tool has to follow data across function boundaries.

@GetMapping("/dashboard/greeting")
@ResponseBody
public String generateDashboard(
        @RequestParam(defaultValue = "Welcome") String greeting) {
    String htmlContent = buildDashboardContent(greeting);
    return htmlContent;
}

private static String buildDashboardContent(String greeting) {
    return "<html><body><h1>Dashboard: " + greeting + "</h1></body></html>";
}

The secure version escapes inside the helper:

@GetMapping("/dashboard/secureGreeting")
@ResponseBody
public String generateSecureDashboard(
        @RequestParam(defaultValue = "Welcome") String greeting) {
    String htmlContent = buildSecureDashboardContent(greeting);
    return htmlContent;
}

private static String buildSecureDashboardContent(String greeting) {
    return "<html><body><h1>Dashboard: " +
            HtmlUtils.htmlEscape(greeting) + "</h1></body></html>";
}

This is where the tools separate. Semgrep CE does not model what happens inside the callee. By default it assumes a call on tainted arguments returns tainted data, which catches the vulnerable version but also flags the secure one. It can instead be configured to trust callees, which clears the false positive but misses the real bug. Either way, it gets one of the two versions wrong. Semgrep Code inspects the callee’s body and handles both correctly.

Results:

⚠️ Semgrep CE: Can either produce false positives or false negatives — cannot see inside the callee.
✅ Semgrep Code: Correctly handles both vulnerable and secure code.
✅ CodeQL and ✅ OpenTaint: Correctly handle both vulnerable and secure code.

From this point, the prose follows Semgrep Code, since inter-procedural analysis is essential. Semgrep CE was still run on the remaining cases — its results appear in the summary table, where it matches Semgrep Code from here on.

Field sensitivity — constructor chains

Here a notification system wraps user-supplied content inside a structured template — a tree of objects (template → body → content → text). The controller below builds that tree from a query parameter and returns the deepest field. Tracking the input through this construction requires field sensitivity: the analyzer has to know which field of which object holds the tainted value.

@GetMapping("/notifications/template")
@ResponseBody
public String generateTemplate(
        @RequestParam(defaultValue = "New Message") String content) {
    Profile.MessageTemplate template = new Profile.MessageTemplate(content);
    return template.body.content.text;
}

The HTML content is built through a chain of constructors:

public MessageTemplate(String text) {
    this.body = new MessageBody("<html>" + text + "</html>");
}

public MessageBody(String text) {
    this.content = new MessageContent("<body>" + text + "</body>");
}

public MessageContent(String text) {
    this.text = "<h1>Notification: " + text + "</h1>";
}

All three tools detect this first constructor-based example. This case has two versions — the three-deep chain above and a six-deep one below — and each has a secure variant that reads secureText instead of text. The MessageContent constructor escapes the value with HtmlUtils.htmlEscape before storing it:

public class MessageContent {
    public String text;
    public String secureText;

    public MessageContent(String text) {
        this.text = "<h1>Notification: " + text + "</h1>";
        this.secureText = "<h1>Notification: " + HtmlUtils.htmlEscape(text) + "</h1>";
    }
}

// NotificationController.java — secure version
@GetMapping("/notifications/secureTemplate")
@ResponseBody
public String generateSecureTemplate(
        @RequestParam(defaultValue = "New Message") String content) {
    Profile.MessageTemplate template = new Profile.MessageTemplate(content);
    return template.body.content.secureText;
}

The next variant extends the field chain further:

@GetMapping("/notifications/generate")
@ResponseBody
public String generateNotification(
        @RequestParam(defaultValue = "New Message") String content) {
    Profile.UserProfile profile = new Profile.UserProfile(content);

    return profile.settings.config.template.body.content.text;
}

And its secure counterpart:

// NotificationController.java — secure version
@GetMapping("/notifications/secureGenerate")
@ResponseBody
public String generateSecureNotification(
        @RequestParam(defaultValue = "New Message") String content) {
    Profile.UserProfile profile = new Profile.UserProfile(content);

    return profile.settings.config.template.body.content.secureText;
}

Here the tools diverge. Semgrep Code and OpenTaint track the deeper field chain. CodeQL does not report the vulnerability — its taint-tracking model does not propagate through field stores and loads on heap objects beyond a limited depth, so the six-deep accessor chain exceeds what its default java/xss query tracks. On the secure variants, Semgrep Code produces false positives — it flags the sanitized secureText field as vulnerable, unable to distinguish it from the tainted text field on the same object.

Results:

⚠️ Semgrep Code: Detects both simple and complex vulnerable versions but produces false positives on secure variants.
⚠️ CodeQL: Handles the simple version but misses the complex one.
✅ OpenTaint: Correctly handles both versions, filtering out secure variants.

Pointer analysis — builder pattern with virtual dispatch

The final case uses a builder pattern. Method chaining returns the same instance, and a field assigned in one call is read in the next — the analyzer must carry the field across the chained call to keep the value reachable at the sink. The pointer analysis named in the table comes into play in the later variants, where what a reference actually points at decides the verdict.

@GetMapping("/message/display")
@ResponseBody
public String displayMessage(
        @RequestParam(defaultValue = "Welcome") String message) {
    String page = new HtmlPageBuilder().message(message).buildPage();
    return page;
}

The builder class stores the message in a field and constructs the response:

private String message = "";

public HtmlPageBuilder message(String message) {
    this.message = message;
    return this;
}

public String buildPage() {
    return "<html><body><h1>Profile Message: " + message + "</h1></body></html>";
}

CodeQL and OpenTaint detect the vulnerability. Semgrep Code does not — its analysis model doesn’t carry the field assigned in .message() across the chained call to .buildPage().

The next variant adds an interface-based formatter:

@GetMapping("/message/format")
@ResponseBody
public String formatMessage(
        @RequestParam(defaultValue = "Welcome") String message) {
    String page = new HtmlPageBuilder().message(message)
            .format(new DefaultFormatter()).buildPage();
    return page;
}

The formatter interface adds another analytical requirement: the analyzer must continue dataflow through a function that takes an interface parameter and invokes a virtual method on it.

public HtmlPageBuilder format(IFormatter formatter) {
    this.message = formatter.format(this.message);
    return this;
}

public class DefaultFormatter implements IFormatter {
    @Override
    public String format(String value) {
        return value; // No actual formatting
    }
}

Detecting the vulnerability does not require resolving which concrete implementation of format() runs — assuming any IFormatter implementation may pass the value through is enough. What the analyzer must do is follow formatter.format(this.message) instead of treating the virtual call as opaque.

Semgrep Code and CodeQL miss the interface-based case. OpenTaint reports it.

The secure version uses an EscapeFormatter that sanitizes the input:

@GetMapping("/message/escape")
@ResponseBody
public String escapeMessage(
        @RequestParam(defaultValue = "Welcome") String message) {
    String page = new HtmlPageBuilder().message(message)
            .format(new EscapeFormatter()).buildPage();
    return page;
}

Filtering this case is where pointer analysis really matters. The analyzer has to know specifically that the formatter parameter holds an EscapeFormatter instance — not just some IFormatter — so the virtual call resolves to the sanitizer rather than to DefaultFormatter, which returns its input unchanged. Without that precision, the analyzer has to consider every IFormatter implementation as possible — including DefaultFormatter — and so flags the secure variant as a false positive. OpenTaint correctly identifies this as safe.

Results:

❌ Semgrep Code: Misses both builder variants.
⚠️ CodeQL: Handles the simple builder but misses the interface-based version.
✅ OpenTaint: Detects both patterns, resolves virtual dispatch, and correctly filters the secure EscapeFormatter variant.

Results summary

Test Case	Semgrep CE	Semgrep Code	CodeQL	OpenTaint
1. Direct return	✅ Pattern ✅ Taint	✅ Pattern ✅ Taint	✅ Built-in	✅ Pattern ✅ Taint
2. Local variable assignment	❌ Pattern ✅ Taint	❌ Pattern ✅ Taint	✅ Built-in	✅ Pattern ✅ Taint
3. Inter-procedural flow	❌ Pattern ⚠️ Taint	❌ Pattern ✅ Taint	✅ Built-in	✅ Pattern ✅ Taint
4. Field sensitivity — constructor chains	❌ Pattern ⚠️ Taint	❌ Pattern ⚠️ Taint	⚠️ Built-in	✅ Pattern ✅ Taint
5. Pointer analysis — builder pattern with virtual dispatch	❌ Pattern ❌ Taint	❌ Pattern ❌ Taint	⚠️ Built-in	✅ Pattern ✅ Taint

Legend

✅ Correctly detected vulnerability and identified secure code.
❌ Missed the vulnerability.
⚠️ Partial success or inconsistent results across complexity variants.

Takeaways

Each tool plateaus at a different depth of analysis:

Semgrep CE handles syntax matching and local taint tracking but stops at function boundaries.
Semgrep Code extends through inter-procedural analysis and deep field chains, but it cannot tell a sanitized field from a tainted one on the same object and does not follow builder patterns or virtual dispatch.
CodeQL covers most cases but hits its limits at deep field chains and virtual calls.
OpenTaint tracks data through all five cases — including builder state, constructor chains, and interface dispatch — using the same pattern rules throughout.

What separates the tools here isn’t the rules — all three express roughly the same intent — untrusted input reaching a dangerous use. It’s how far each engine carries a tracked value on its own. In OpenTaint the same pattern rule that catches a value returned directly also catches one routed through a builder. The assignments, inter-procedural calls, field state, and virtual dispatch in between are resolved by the engine, not spelled out in the rule.

Real codebases are full of these patterns. As code grows it adds helpers, builders, persistence layers, and interface calls, and each one is another place a scanner can lose the value it is tracking. The more layers there are, the more a tool misses. This is why, over time, the engine matters more than the rules. A rule that says what to look for and leaves the how of tracking to the engine is the one that keeps working as the code gets more complex.

Scope

The test is narrow by design. A single common vulnerability and deliberately simple rules take rule quality out of the equation — the only variable left is how far each engine can carry a value. That is also all the results measure. They say nothing about language coverage or performance on a large codebase.

All five cases are runnable end-to-end in the java-spring-demo project. For a deeper look at what Spring-specific data flows OpenTaint can model — dependency injection, JPA persistence, and cross-endpoint tracking — see Taint Analysis for Spring: Security Beyond Syntax.

To try OpenTaint on your own project, see the quick start guide.