Security in the pipeline and in AI features
Automated scanners in CI catch whole classes of flaws before they ship, but only if the build fails for real risks and stays quiet otherwise. This lesson sets up that pipeline, then turns to the newest attack surface in the Partner app: an LLM assistant that reads orders, product text and uploaded invoices, and can be talked into misbehaving by any of them.
By the end you can
- Say what SAST, SCA, secret scanning, DAST and IaC scanning each find and miss, and where each one runs
- Write a build gate that fails only for new high or critical findings with a fix, and honours suppressions that have an owner, a reason and an expiry date
- Explain direct and indirect prompt injection and why filtering prompts cannot stop it
- Design an LLM feature so a fully tricked model still cannot read another shop's data or move money
- Test an AI feature with an eval set of adversarial cases that runs in CI
The everyday picture
Think of a busy restaurant kitchen. Different checks happen at different moments. Someone reads each new recipe before it goes on the menu and spots the one that leaves chicken undercooked. Someone checks the labels on supplier deliveries against the latest food recall list. A supervisor tastes plates at the pass, the counter where finished dishes wait to go out. A fire inspector looks at the gas pipes once a quarter. Each check catches problems the others cannot see: reading a recipe tells you nothing about a recalled batch of flour.
Now picture the smoke alarm that goes off every time someone makes toast. Within a week somebody takes the battery out, and the kitchen is less safe than if the alarm had been quieter. Security scanners in a build pipeline behave the same way. A scanner that fails every build for things nobody can fix teaches developers to ignore it. The skill is choosing which checks run where, and making the loud ones loud only for real fires.
Five scanners, five blind spots
Each kind of automated security test reads a different thing, so each has a different blind spot. You need several, and you need to know what none of them covers.
| Check | Reads | Finds well | Misses |
|---|---|---|---|
| SAST | Your source code, without running it | Untrusted input flowing into a SQL string, an HTML page or a shell command; weak crypto calls; unsafe deserialisation | Missing access checks (it cannot know shop A must not read shop B's invoice), flaws in business logic, anything decided by configuration at runtime |
| SCA | Your lockfile and manifest | Packages with a published advisory, and which version fixes it; licence problems | Flaws nobody has reported yet; whether your code ever calls the vulnerable function |
| Secret scanning | Every commit, including old history | Keys and tokens that match known formats, before or after they are pushed | Custom secret formats; a secret already copied somewhere else |
| DAST | A running test deployment, over HTTP | Missing security headers, cookies without the Secure flag, input reflected unencoded, verbose error pages, exposed admin paths | Pages it cannot reach (behind a login it was not given), the cause in the code, and it is slow |
| IaC scanning | Terraform, Kubernetes manifests, Dockerfiles | A public storage bucket, a container running as root, a database open to the internet | Changes made by hand in the cloud console that never went through the files |
Two gaps matter most. First, no scanner finds broken access control reliably, and that is the number-one web risk you met in Broken access control. Only tests that log in as two different shops and try to cross over, plus code review and threat modelling (Threat modelling a feature), catch it. Second, SCA results change without anyone touching the code. A library that was clean on Monday gets an advisory on Thursday, so a scan that only runs on pull requests will never tell you. Supply chain and secrets covered lockfiles and secret handling; here you decide when the scanners run.
The rule for placement is speed and signal. Fast checks with few false alarms run on every pull request and can block a merge: secret scanning, SAST on the changed code, the SCA difference (new or upgraded packages only) and IaC scanning, all in a few minutes. Slow checks run on a schedule against main and a test deployment: a full DAST scan and an SCA rescan of everything every night. Before a release, a gate checks that nothing critical is open past its fix-by date. If you have not built a workflow before, GitHub Actions: continuous integration covers events, jobs and schedules, and Deploying and securing with GitHub Actions covers GitHub's own scanning features.
Quick check
Six months ago a developer committed a cloud storage key to the Partner API repository and deleted it in the next commit. Which check finds it today?
Deleting a file does not delete it from history, so the key is still readable by anyone with the repository. Secret scanning reads every commit. SAST reads the current code, where the key no longer is. And finding it is only half the job: the key must be revoked and replaced, because you cannot know who copied it.