pipeline
An automated sequence of steps, such as build, test, scan and publish, that runs on every change.
In more detail
Pipelines run automatically, usually in a CI service, so every change gets the same checks. Anything a pipeline runs must be written in files with versions pinned, because nobody is there to type commands.
Examples
A pipeline can act as the Ansible control node, linting and testing roles and then applying playbooks.The course pipeline runs a lint gate, then unit, integration and end-to-end tests, then builds one artifact that later stages promote. A pipeline owns validation and provenance; a GitOps controller owns convergence.cd ci && bash pipeline.shA token pipeline compiles design tokens for every platform; a CI pipeline runs accessibility and visual tests before a change can merge.In Terraform: A Terraform pipeline typically formats, initialises, validates, tests, plans, checks policy and, after approval, applies. It must never stop to ask a question, so it passes -input=false and applies saved plans.In this course: A quality gate is expressed as pipeline jobs: lint, types, unit tests and a focused integration suite, each with a clear failure message. In continuous integration the pipeline runs on every push and pull request.Related terms
- continuous integration — Automatically building and testing every change as soon as it is pushed, so problems are caught early.
- control node — The machine where Ansible is installed and where you run its commands; it reaches out to every other machine it manages.
- pinning — Fixing an exact version or digest of something you depend on so it cannot change without you noticing.
- quality gate — A check, or set of checks, that a change must pass before it can move on, such as being merged or promoted to the next pipeline stage.
- artifact promotion — Moving the exact artifact that passed testing on to the next environment, instead of building it again.
- GitOps — Operating a system by keeping its desired state in Git and letting a controller continuously make the live system match it.
- trigger — The event that starts a pipeline run, most often somebody pushing a commit.
- runner — A machine running the runner application that picks up a queued job, runs its steps and reports the result.
- artifact — A file produced by a build, such as a compiled program or an image, that is stored and passed on to later steps.
- workflow (GitHub Actions) — A YAML file in the .github/workflows folder of a repository that names its triggers and the jobs to run.
- image promotion — Moving the exact same, already-tested image through environments, such as staging to production, without rebuilding it.
- secret — Any value that grants access, such as a password, access token or private key, and must be kept out of code and version control.
- security gate — An automated check in a pipeline that stops a change going further when it fails, usually by exiting with a non-zero code.
- Style Dictionary — An open-source build tool that reads design token files and generates code for each platform, such as CSS, TypeScript, Swift and Android resources.
- Turborepo — A build tool for JavaScript monorepos that runs package tasks in dependency order, in parallel, and skips work whose inputs have not changed.
- continuous delivery — Keeping every change that passes the pipeline in a state where it could be released to production at any moment.
- deployable — Something built and released as one unit, such as a service or a monolith.
- ingestion — Accepting incoming data and getting it safely into storage.
- -input=false — The option that stops Terraform from asking any interactive question, so an automated run fails instead of hanging forever.
- saved plan — A plan written to a file with -out, which terraform apply can later carry out exactly as reviewed.
- GitHub Actions — GitHub's built-in automation service that runs workflows, written as YAML files in a repository, on runners when events happen.
- noEmit — The compiler option that makes tsc check types without writing any output files.
Taught in
- Ansible Architecture And The Control Node — Ansible Configuration Management at Scale
- Installing Ansible — Ansible Configuration Management at Scale
- Your First Ad Hoc Command — Ansible Configuration Management at Scale
- Dynamic Inventory — Ansible Configuration Management at Scale
- Two Hundred Accounts And One Bad Deploy — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Surviving A Hundred And Eighty Minutes — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- CodePipeline: Stages, Actions And The Key Nobody Shared — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- CodeBuild: The Build That Passed With Failing Tests — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Artifacts: Where Packages And Images Live, And Die — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- CodeDeploy: Hooks That Never Ran — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- CloudFormation In Depth: Replace Or Update? — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- StackSets: Eighty Failures Before Lunch — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Testing Resilience: The Experiment That Would Have Been The Outage — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Monitoring As Code: The Alarm That Watched Nothing — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- EventBridge Rules: The Rule That Matched Nothing — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Remediation As Code: A Public Bucket Appears — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Troubleshooting Failed Deployments: Read The First Failure — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Identity At Scale: The Boundary That Blocked The Admin — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Auditing: Who Opened Port Twenty Two? — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Pipeline Security: The Build Role That Could Do Anything — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Long-Stem Clinic: When Two Answers Both Work — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- The Full Mock And Your Review Plan — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- What Is a Pipeline? — Continuous Integration & Continuous Delivery Pipelines
- DORA Metrics: Measuring Your Pipeline — Continuous Integration & Continuous Delivery Pipelines
- The Push Model vs. The Pull Model — Continuous Integration & Continuous Delivery Pipelines
- Feature Flags and DORA Metrics — Continuous Integration & Continuous Delivery Pipelines
- ServiceAccounts And Tokens — Certified Kubernetes Application Developer (CKAD) Exam Prep
- Workload Identity Federation: Eliminating Static Keys — Cloud Security & DevSecOps Engineering
- Cross-Account AssumeRole Chains & Temporary STS — Cloud Security & DevSecOps Engineering
- Shift-Left Security Principles & Automated Pipeline Gates — Cloud Security & DevSecOps Engineering
- Pre-Commit Secret Scanning with Gitleaks & Entropy — Cloud Security & DevSecOps Engineering
- Static Application Security Testing with Semgrep — Cloud Security & DevSecOps Engineering
- Software Composition Analysis with Trivy — Cloud Security & DevSecOps Engineering
- Supply Chain Security: SBOMs & Cosign Signing — Cloud Security & DevSecOps Engineering
- Runtime Anomaly Detection with Falco & eBPF — Cloud Security & DevSecOps Engineering
- Static IaC Scanning: Linting Terraform with Checkov — Cloud Security & DevSecOps Engineering
- Policy-as-Code Architecture: OPA & Rego Syntax — Cloud Security & DevSecOps Engineering
- Enforcing OPA Guardrails on Terraform Plan Payloads — Cloud Security & DevSecOps Engineering
- Continuous IaC Drift Detection & State Protection — Cloud Security & DevSecOps Engineering
- Cloud Security Posture Management & Remediation — Cloud Security & DevSecOps Engineering
- Token Transformation: Style Dictionary & Multi-Platform Outputs — Design Systems & Component Engineering
- Visual Contrast Auditing: Color Blindness & Reduced Motion — Design Systems & Component Engineering
- Visual Regression Testing: Playwright & Snapshot Differencing — Design Systems & Component Engineering
- Continuous Integration — DevOps & Site Reliability Engineering
- Testing Strategies And The Test Pyramid — DevOps & Site Reliability Engineering
- Building And Versioning The Artifact — DevOps & Site Reliability Engineering
- Continuous Delivery Versus Continuous Deployment — DevOps & Site Reliability Engineering
- Internal Developer Platforms — DevOps & Site Reliability Engineering
- Infrastructure As Code Context — DevOps & Site Reliability Engineering
- Pipelines And GitOps Context — DevOps & Site Reliability Engineering
- Docker, OCI, The Engine And The CLI — Docker Architecture & Production Containers
- RUN, SHELL And Efficient Layer Caching — Docker Architecture & Production Containers
- ARG, ENV, LABEL, EXPOSE And VOLUME — Docker Architecture & Production Containers
- Multi Stage Builds, Targets And Build Secrets — Docker Architecture & Production Containers
- Tags, Digests And Reproducible Image References — Docker Architecture & Production Containers
- Authenticate, Tag, Push And Pull From A Registry — Docker Architecture & Production Containers
- Scan, Patch And Inspect The Software Supply Chain — Docker Architecture & Production Containers
- Development Overrides, Hot Reloading And Debuggers — Docker Architecture & Production Containers
- Integration Tests, Test Data And Compose Cleanup — Docker Architecture & Production Containers
- Diagnose Crashes, Resource Limits And Unhealthy Containers — Docker Architecture & Production Containers
- Build, Test And Promote The Same Image In CI — Docker Architecture & Production Containers
- From Local Containers To Kubernetes — Docker Architecture & Production Containers
- Monorepo Architecture: Workspaces & Turborepo — Production Full-Stack Engineering
- Unified Build Pipelines: Caching & Dependency Graphs — Production Full-Stack Engineering
- Continuous Delivery: Blue-Green & Canary Pipelines — Production Full-Stack Engineering
- The Merge Request That Deployed To Production — GitLab Certified CI/CD Associate Exam Prep
- How The Exam Works: Fifty Questions, No Notes — GitLab Certified CI/CD Associate Exam Prep
- Continuous Integration, Delivery And Deployment: One Keyword Apart — GitLab Certified CI/CD Associate Exam Prep
- From Push To Runner: How GitLab Starts A Pipeline — GitLab Certified CI/CD Associate Exam Prep
- Anatomy Of .gitlab-ci.yml, And Three Ways To Check It — GitLab Certified CI/CD Associate Exam Prep
- Jobs And Stages: Who Waits, Who Runs, Who Gets Skipped — GitLab Certified CI/CD Associate Exam Prep
- Rules, Workflow Rules And The Legacy only And except — GitLab Certified CI/CD Associate Exam Prep
- Reuse With extends And include, And What Survives The Merge — GitLab Certified CI/CD Associate Exam Prep
- Stages Versus Needs: Building The Pipeline As A Graph — GitLab Certified CI/CD Associate Exam Prep
- Parent, Child And Multi-Project Pipelines — GitLab Certified CI/CD Associate Exam Prep
- Artifacts: Which Files Reach Which Jobs — GitLab Certified CI/CD Associate Exam Prep
- Caching: Dependencies That Survive Between Pipelines — GitLab Certified CI/CD Associate Exam Prep
- CI/CD Variables: Where Values Come From, And Which One Wins — GitLab Certified CI/CD Associate Exam Prep
- Environments, Review Apps And Deployments — GitLab Certified CI/CD Associate Exam Prep
- The Container Registry: Build, Tag, Push, Clean Up — GitLab Certified CI/CD Associate Exam Prep
- Package Registries: Publishing With The Job Token — GitLab Certified CI/CD Associate Exam Prep
- Adding GitLab's Security Scanners To A Pipeline — GitLab Certified CI/CD Associate Exam Prep
- Reading Security Results: The Merge Request And The Vulnerability Report — GitLab Certified CI/CD Associate Exam Prep
- Registering Runners And Routing Jobs With Tags — GitLab Certified CI/CD Associate Exam Prep
- Debugging Pipelines: The Graph, The Errors, The Logs — GitLab Certified CI/CD Associate Exam Prep
- Faster Pipelines: Minutes You Can Win Back — GitLab Certified CI/CD Associate Exam Prep
- Pipeline Clinic: Five Tickets, Five Fixes — GitLab Certified CI/CD Associate Exam Prep
- Recall Sprint: Knowing It Cold, With No Docs And No AI — GitLab Certified CI/CD Associate Exam Prep
- Final Mock And Review: One Question Short — GitLab Certified CI/CD Associate Exam Prep
- Installing the Go Toolchain — Concurrent Systems Programming with Go
- Go Workspaces and Modules — Concurrent Systems Programming with Go
- Your First Program and go run — Concurrent Systems Programming with Go
- Variables, Typing, and Zero Values — Concurrent Systems Programming with Go
- Errors as Values — Concurrent Systems Programming with Go
- Files and the os Package — Concurrent Systems Programming with Go
- Parsing Arguments with flag — Concurrent Systems Programming with Go
- Exporting Results as JSON — Concurrent Systems Programming with Go
- The Environment Variable Trap — Application Security & Threat Modeling for Engineers
- Dependency Risk And Typosquatting — Application Security & Threat Modeling for Engineers
- Software Bill Of Materials — Application Security & Threat Modeling for Engineers
- Container Security: Minimal Bases And Non Root — Application Security & Threat Modeling for Engineers
- Static Application Security Testing — Application Security & Threat Modeling for Engineers
- Secret Scanning — Application Security & Threat Modeling for Engineers
- How To Stop Drowning In Findings — Application Security & Threat Modeling for Engineers
- Retries, Backoff And Jitter — Scalable System Design & Distributed Architecture
- Monolith Versus Services: The Honest Trade Offs — Scalable System Design & Distributed Architecture
- Observability As A Design Input — Scalable System Design & Distributed Architecture
- Worked Design Three: A Metrics Ingestion Pipeline — Scalable System Design & Distributed Architecture
- Worked Design Three: Scaling The Metrics Pipeline — Scalable System Design & Distributed Architecture
- Installing Terraform And Meeting The CLI — Infrastructure as Code with Terraform
- Initialising The Working Directory — Infrastructure as Code with Terraform
- Predicting Changes, Then Applying Them — Infrastructure as Code with Terraform
- Parameterising With Input Variables — Infrastructure as Code with Terraform
- Running Terraform In CI — Infrastructure as Code with Terraform
- Native Testing With Tftest Files — Infrastructure as Code with Terraform
- Provider Mocking For Tests — Infrastructure as Code with Terraform
- Policy As Code With OPA — Infrastructure as Code with Terraform
- Releasing The Real Cloud Artifact — Infrastructure as Code with Terraform
- What To Automate Away — Automated Testing, TDD & Quality Engineering
- Why TypeScript: Contracts, Erased Types And Soundness — Production TypeScript
- The Compiler Pipeline: tsc, AST And Code Emission — Production TypeScript
- Type Annotations, Type Inference And Contextual Typing — Production TypeScript
- Verify Your Environment: Running tsc And Bun TypeScript — Production TypeScript
- Tuples: Fixed Length, Labeled Elements And Variadic Tuples — Production TypeScript
- Assertion Signatures: asserts condition And Invariants — Production TypeScript
- The Spine Artifact: Generating Typed Models From OpenAPI — Production TypeScript
- The Plan That Wanted To Destroy Production — HashiCorp Terraform Associate (004) Exam Prep
- Versions, The Lock File And Provider Aliases — HashiCorp Terraform Associate (004) Exam Prep
- Write, Plan, Apply, And What Init Prepares — HashiCorp Terraform Associate (004) Exam Prep
- Reading A Plan, Saving It, Applying It — HashiCorp Terraform Associate (004) Exam Prep
- Validate, Format, Destroy, And The Flags That Bend A Plan — HashiCorp Terraform Associate (004) Exam Prep
- Input Variables, Outputs, And Which Value Wins — HashiCorp Terraform Associate (004) Exam Prep
- Custom Conditions: Validation, Pre And Postconditions, Checks — HashiCorp Terraform Associate (004) Exam Prep
- Module Versions: Pin, Upgrade, Re-Initialise — HashiCorp Terraform Associate (004) Exam Prep
References
- Ansible Lint: usage
- GitHub Docs: understanding GitHub Actions
- Docker Build: continuous integration
- AWS: What is continuous delivery?
- GitHub Actions: continuous integration
- Turborepo: Configuring tasks
- NIST SP 800-218: Secure Software Development Framework
- Google SRE book: Data Processing Pipelines
- Running Terraform in automation
- TSConfig: noEmit
Certification courses
- AWS DevOps Engineer Professional course and practice exam: DOP-C02
- CKAD course and practice exam: Certified Kubernetes Application Developer
- GitLab CI/CD Associate course and practice exam: GitLab Certified CI/CD Associate
- Terraform Associate course and practice exam: HashiCorp Terraform Associate (004)