Scanning AI-Generated Code in Pipelines: A Gated PR Intake
Scanning AI-generated code in pipelines: a gated pull request intake for apps built in Lovable, Cursor or Claude Code, before they reach company data.
Microsoft already told you whose job this is
Microsoft’s documentation for Copilot Managed Runtime (preview) states plainly who owns this problem. Externally built artifacts “bypass the Microsoft-managed build system, so your organization is responsible for validating them and securing their build pipeline” (Microsoft Learn). Business users are building apps in Lovable, Replit, Cursor, Claude Code and v0, and they want those apps connected to company data. That sentence assigns the job of scanning AI-generated code in pipelines to you.
The first risk to remove has a name. “The number one risk at the minute is hard-coded secrets being uploaded through vibe-coded applications to GitHub, and then providing a route in,” Pete Shoard, chief of research for cybersecurity at Gartner, told Computerworld.
My position: let people build with AI, and make the pull request the only door into company data. The tools are not the problem. An app that reaches a database without anyone reading its code is the problem.
Every outside-built app gets a repo and a PR before it gets a connection string.
Why blocking the tools fails and gating the output works
Blocking a login page is easy. Stopping someone from running Cursor or Claude Code against a folder on their laptop is not, and the attempt mostly teaches people to stop asking IT.
Even a Microsoft-integrated path leaves the source outside your reach. Microsoft’s Lovable setup page lists “The Lovable project source”, “Project chat history” and “Lovable workspace and connection metadata” as content that remains outside your Microsoft tenant (Microsoft Learn). If the code never lands in your repo, you cannot scan it.
Cleanup carries a practitioner caveat too. “Revoking the Lovable integration prevents future access, but it doesn’t automatically remove apps that were already deployed.” To remove them, admins go to Apps > All apps in the Microsoft 365 admin center and block or delete the deployed apps.
So turning a connection off does not undo what is already running. That is a good reason to decide the entry path before the first app arrives, not after the tenth.
The gaps that worry security leads all have a matching control. Secret scanning catches hard-coded secrets. Dependency alerts catch known-vulnerable packages, and a named reviewer catches missing authorization.
None of these controls requires banning a tool. Each one requires the code to sit somewhere the checks can run.
Governance happens at intake, not at the tool’s login screen.
Gate 0: Intake through source control, no exceptions
I recommend requiring GitHub Enterprise Cloud or Azure DevOps Services Git for every intake repo. Two documented reasons support this. Copilot Managed Runtime (preview) “supports external repositories owned by GitHub Enterprise Cloud organizations” (Learn). DevOps security in Defender for Cloud supports Azure DevOps Services, GitHub Enterprise Cloud and GitLab SaaS (Learn).
The scanners themselves do not require Enterprise Cloud. GitHub says “You must be on a GitHub Team or GitHub Enterprise plan in order to purchase GitHub Code Security or GitHub Secret Protection” (GitHub Docs). My recommendation is about the deploy target and the central console, not about where the scanners run.
Two edge cases apply. DevOps security in Defender for Cloud does not currently support GitHub Enterprise Cloud instances configured with data residency (Learn, support and prerequisites). GitHub Advanced Security for Azure DevOps “is only available for Azure DevOps Services and specifically for code Git repositories” (Learn), so Azure DevOps Server and TFVC are out.
On licensing, Dependabot alerts come with all GitHub plans. On private and internal repositories, secret scanning and push protection need GitHub Secret Protection, and code scanning needs GitHub Code Security. Both are sold on Team or Enterprise (Secret scanning, Code scanning).
On 2026-11-06 the GitHub list prices were $19 USD for Secret Protection and $30 USD for Code Security, per active committer/month (GitHub). A committer counts as active “if one of their commits has been pushed to the repository within the last 90 days” (GitHub Docs). The Azure DevOps pricing page lists $19 for Secret Protection and $30 for Code Security per committer per month. It adds: “Prices are estimates only and are not intended as actual price quotes.”
Two handoffs are banned: zip files, and “I deployed it straight from the tool.” The app arrives as a pull request into a repo you own, or it does not arrive.
Here is how I recommend the intake repo works in practice. IT creates the repo, and the business builder pushes their code into a branch and opens a pull request. The repo holds no connection strings, and no environment that can read company data deploys from it yet.
Treat the intake repo as a quarantine. Merging into it means the code has passed the merge checks, not that the app is approved. Data access is a separate grant that comes at the very end, once every gate is clear.
That split matters because some checks only see the default branch. Gate 2 depends on it.
Intake also connects directly to shadow AI governance. The app inventory you build there becomes your repo list here.
No repo, no data access.
The gate overview table: scanning AI-generated code in pipelines, in order
The gate order below is my recommendation, built on what each platform documents. Read the “Enforced by” column carefully. “Intake policy” means a written rule your reviewers apply: the app does not move forward while that item is open.
Note the two checkpoints. Some items hold the merge into the quarantined intake repo. Others hold the data connection, because they only report after code reaches the default branch.
| Gate | What it checks | GitHub control | Azure DevOps control | Enforced by | Caveat |
|---|---|---|---|---|---|
| 1 | Hard-coded credentials, new and historical | Secret scanning plus push protection (Secret Protection) | Secret Protection for Azure DevOps: secret scanning with push protection | Push protection blocks pushes that contain secrets before they reach the repository. Alerts on existing secrets: intake policy | GitHub: push protection is disabled by default, and write users can bypass it with a reason unless restricted. Azure DevOps: push protection only evaluates pushes after it is enabled |
| 2 | Known-vulnerable packages | Dependabot alerts (all plans) | Code Security for Azure DevOps: dependency scanning | GitHub: intake policy, after merge. Alerts appear on the default branch, so the intake repo stays cut off from company data until its alert list is clean. Azure DevOps: Advanced Security status checks block merges when vulnerabilities are detected | Dependabot scans the default branch only; only GitHub-reviewed advisories alert; lock files must be current |
| 2 | Vulnerable code patterns | Code scanning: CodeQL, plus SARIF from third-party tools | Code Security for Azure DevOps: CodeQL code scanning | GitHub: intake policy. Azure DevOps: Advanced Security status checks block merges when vulnerabilities are detected | CodeQL does not support PHP or Scala |
| Signal (not a gate) | Vulnerabilities in PR changes for languages CodeQL misses | AI Scan for pull requests | None | Advisory, cannot block merges | Covers examples such as PHP and Terraform (HCL) |
| Signal (not a gate) | Code findings across connected repos | Defender for Cloud agentless code scanning | Defender for Cloud agentless code scanning | Can't break builds | Preview, default branch only, can't break builds |
| 3 | Row-level policy, anonymous access, server-side authorization | Named reviewer with checklist | Named reviewer with checklist | Intake policy | Scanners do not answer these questions |
| 4 | Data flows, auth boundaries, earlier flags | Scoped human review | Scoped human review | Intake policy | Recommended scope, not a full audit |
AI Scan for pull requests and agentless code scanning are signals, not gates. Every other row is a gate. The platform enforces it where the docs say it blocks, and your intake policy enforces it everywhere else.
Gate 1: Secrets first, before anything else runs
Gate 1 has two parts, and you need both. Repository secret scanning “scans your entire Git history on all branches of your repository for hardcoded credentials” (GitHub Docs). Secret scanning push protection “blocks pushes that contain secrets before they reach your repository” (GitHub Docs).
The Azure DevOps docs explain why both matter: “Push protection only evaluates pushes that occur after the feature is enabled. Repository-level secret scanning detects and alerts on existing secrets, including those from historical commits.” An imported app arrives with its history attached.
Push protection never sees that history. Repository scanning does. This is exactly the case for outside-built apps, where weeks of prompting and iteration can land in a single import.
Secrets go first for a practical reason. A leaked credential is live the moment it is pushed, while a vulnerable package or a weak policy only matters once the app runs against real data. Clearing secrets before anything else also means later reviewers never handle a working key.
Now for the caveat that catches teams out. On GitHub, repository push protection “Is disabled by default” and “Requires GitHub Secret Protection to be enabled.” Also, “by default, anyone with write access to the repository can bypass push protection by specifying a bypass reason.”
Restrict who can bypass on intake repos. Do not confuse this with push protection for users. That one is on by default, but it only stops pushes to public repositories on GitHub.com.
When a secret turns up, follow GitHub’s order: “rotate the affected credential immediately to prevent unauthorized access.” Rewriting history “is time-intensive and often unnecessary if you’ve already revoked the credential.” Rotate first, and skip the afternoon of history surgery.
The standard behind this gate is OWASP ASVS 5.0.0 requirement 13.3.1. It calls for a secrets management solution such as a key vault, and states: “Secrets must not be included in application source code or included in build artifacts.” It is a Level 2 requirement, and ASVS says of Level 2: “Most applications should be striving to achieve this level of security.”
Azure DevOps teams should note one change: “As of September 20, 2023, the secrets scanning (CredScan) tool within the Microsoft Security DevOps (MSDO) Extension for Azure DevOps is deprecated” (Learn). GitHub Advanced Security for Azure DevOps replaces it.
Turn on push protection so new secrets stop at the push. Treat every secret scanning alert as rotate-first, and let it hold the intake PR under your intake policy.
Gate 2: Dependencies and code, before any human reads it
Dependabot alerts come with every GitHub plan. Read the fine print, though. Dependabot “scans your repository’s default branch,” and “Only advisories reviewed by GitHub trigger alerts” (GitHub Docs).
GitHub also tells you to keep manifest and lock files up to date for accurate detection. Alerts fire when a new advisory lands or when “Your dependency graph changes”, so an app keeps generating alerts long after intake.
Because the scan reads the default branch, Dependabot alerts cannot sit on the intake PR. They appear after the merge. I recommend making that sequencing explicit: the merge lands in the quarantined intake repo, and the repo stays cut off from company data until its Dependabot alert list is clean.
Automatic fixes have a narrower scope than alerts. If you enable Dependabot security updates, “Dependabot automatically tries to fix it,” but “security updates are triggered only for dependencies that are specified in a manifest or lock file” (GitHub Docs).
Require the lockfile in the intake PR. An app without a lockfile has not finished arriving.
CodeQL code scanning supports C/C++, C#, Go, Java/Kotlin, JavaScript/TypeScript, Python, Ruby, Rust, Swift and GitHub Actions workflows. GitHub is explicit that “CodeQL does not support languages that are not listed above. This includes, but is not limited to, PHP, Scala, and others” (GitHub Docs).
Fill the gaps with a third-party scanner, since code scanning accepts SARIF output from other tools. You can also run the CodeQL CLI in an external CI system and upload the results to GitHub. Budget for runtime too: “Code scanning uses GitHub Actions, with each workflow run consuming GitHub Actions minutes.”
AI Scan for pull requests covers languages CodeQL misses, such as PHP and Terraform (HCL). GitHub is clear about its limits: “Findings are advisory. They do not fail the Code scanning check and cannot block pull request merges through ruleset requirements” (GitHub Docs). Read its findings, but do not count it as a gate.
GitHub Advanced Security for Azure DevOps gives you a real block. “Advanced Security status checks allow you to block pull requests from being merged when security vulnerabilities are detected.” The CodeQL default setup covers C#, C/C++, Java, JavaScript/TypeScript, Python, Ruby and Rust. For Go and Swift, use advanced setup.
This gate runs before any human reads the code, and the order is deliberate. Scanners are tireless at pattern matching and poor at judgment. Let them clear the mechanical findings so reviewer time goes to the questions only people answer.
The security lead also wants one view across platforms. DevOps security in Defender for Cloud gives “a central console” across Azure DevOps, GitHub and GitLab (Learn). Some capabilities, such as attack path analysis, need the paid Defender CSPM plan.
Defender does not find secrets by itself, though. The exposed-secrets recommendation lists GitHub Advanced Security, or GitHub Advanced Security for Azure DevOps, as a prerequisite. Agentless code scanning stays a signal: it is preview, covers the default branch only, and “Can’t break builds” (Learn).
The ASVS anchors for this gate are 15.2.1 (Level 1), which covers components that have not breached documented update time frames, and 15.1.2 (Level 2), which covers an SBOM-style inventory of third-party libraries. The lockfile requirement gives you a start on that inventory.
On Azure DevOps, turn on Advanced Security status checks so vulnerable code cannot merge. On GitHub, treat open code scanning alerts as must-resolve under the intake policy, and keep the intake repo cut off from company data until its Dependabot alert list is clean.
Gate 3: Access-policy review, read by a named person
NVD record CVE-2025-48757 describes “An insufficient database Row-Level Security policy in Lovable through 2025-04-15” that “allows remote unauthenticated attackers to read or write to arbitrary database tables of generated sites.” Lovable disputes it. The record says so, “because each individual customer of the Lovable platform accepts a responsibility over protecting the data of their application.”
NVD lists the record as Deferred. Set the dispute aside, because the supplier’s own position says the app owner protects the data. The control follows from that: verify row-level security, and never assume it is there.
ASVS sets the bar at Level 1 for both requirements that matter here (V8). Requirement 8.2.2 restricts data-specific access to prevent IDOR and BOLA. Requirement 8.3.1 enforces authorization “at a trusted service layer” and not through “client-side JavaScript.”
The automated scanners in Gates 1 and 2 do not answer these questions. A person reads the policies. Name that person in the intake policy, and make the sign-off something the PR history shows, so nobody approves an access model by default.
No table ships until a named reviewer signs off on its access policy.
Gate 4: A scoped human review, then a controlled deploy
The human pass covers data flows, auth boundaries and everything the earlier gates flagged. That is all it covers, and I recommend keeping it to that scope instead of turning it into a full audit. Set a time box that fits your team. The earlier gates already did the mechanical work, so this review is about judgment.
The deploy target is Copilot Managed Runtime (preview). “External artifact deployment is disallowed by default” (Learn). The externally built artifacts setting “is disabled by default and can be configured for an environment or environment group.” Turn it on only for environments that this pipeline feeds.
The documented path is GitHub Actions with the Copilot Managed Runtime CLI (ms), and that CLI’s command reference is marked preview. Azure DevOps teams should know about the gap. The Learn documentation describes no Azure DevOps Pipelines deploy route to this runtime.
Run Gates 0 to 3 in Azure DevOps if that is your platform. Do not plan around a deploy path that is not documented.
Keep two protection layers separate. The build sandbox restricts network access during Microsoft-managed builds, and Learn says it applies “to apps that use platform-managed repositories, not external GitHub repositories.” Your pipeline’s artifacts never pass through it.
The runtime layer works differently. App code runs in a sandboxed iframe under a Content Security Policy, which together “reduce what malicious code can do at runtime.” Learn adds: “These protections don’t replace reviewing app code and dependencies.”
The Lovable integration is a separate path that an admin configures. Lovable builds the artifact outside the Microsoft-managed build system, and Copilot Managed Runtime (preview) hosts it. That route skips this intake.
My recommendation is that environments using the PR intake do not also enable the Lovable integration. Two doors into the same environment means the stricter one stops mattering. For how review depth should scale with data sensitivity, see Risk-Tiered Agent Governance.
Turn on external artifacts only where your pipeline is the sole way in, and keep the Lovable integration off in those environments.
The repository is the border
Once every outside-built app enters through a PR with secret and dependency checks that must pass, a signed-off access policy and a scoped human pass, the conversation changes. IT stops chasing apps and starts approving them. Scanning AI-generated code in pipelines is how you say yes without losing control.
Have outside-built apps waiting at the door?
If you lead IT, a CoE or security and you have apps from Lovable, Cursor or Claude Code queued up for company data, start with an outside view. Citizen Development Academy looks at what your team has built or plans to build with AI, names the top three risks, and recommends one next step. Book a free 30-minute AI app review with Citizen Development Academy.
Stay in the loop
Get new posts delivered to your inbox. No spam, unsubscribe anytime.
Related articles
Configure Copilot Studio Security Controls for Agents Your Business Users Build
Configure Copilot Studio security controls in the right order: data policies, build access, knowledge, sharing, authentication and monitoring for agents.
Power Platform Environment Groups Reference Setup for AI-Built Apps
A Power Platform environment groups reference setup for AI-built apps: managed environments, group rules, advanced connector policy and routing, in order.