r/Terraform • • 8h ago

AWS Built a maintained replacement for driftctl: refresh-only drift ranked by severity, plus AWS resources not in any state

1 Upvotes

Since driftctl stopped being developed, we were stitching together terraform plan -refresh-only for managed drift and ad-hoc scripts for "what's in the account that isn't in state?". I turned that into one small CLI.

  • cloud-drift-report terraform --dir ./infra runs plan -refresh-only -lock=false (Terraform or OpenTofu) and turns the output into a report ranked by severity: security groups, IAM and KMS changes first, tag-only changes last, each with a short explanation.
  • cloud-drift-report unmanaged --region us-east-1 --state s3://my-state/prod/ lists the account (~170 types via Cloud Control and native list calls), skips AWS defaults, and reports what no state file covers.

Read-only, Apache-2.0, pip install cloud-drift-report. There's a GitHub Action too.

What I'd most like feedback on: resources it wrongly reports as unmanaged. The matching is by ID/ARN/name, and some resource types record odd IDs in state.

Repo: https://github.com/orchesterra/cloud-drift-report

Disclosure: I built this at OrchesTerra, where we also sell a hosted drift platform. The CLI is free, open source and works on its own.


r/Terraform • • 4h ago

Discussion Agent written Terraform PRs keep fighting manual changes in the console

0 Upvotes

We manage about 400 resources in Terraform across 3 AWS accounts. Agents write most of our infra PRs now, and they write them against the code, not against what's actually deployed.

Last month someone had raised an RDS instance size in the console during an incident and never put it back in code. Two weeks later an agent PR for an unrelated tag change went through, the plan showed the instance size going back down, it was one line in a 60 line plan, and nobody noticed. The database got resized during business hours.

coderabbit reviews the HCL and it's useful, it caught a hardcoded secret in a variable default last month. But drift between code and reality isn't in the diff, it's only in the plan, and long plans don't get read carefully.

What we're trying now:

- a scheduled drift check that opens an issue when real state differs from code

- failing CI if a plan modifies or replaces anything in our "do not touch" list

- posting a short plan summary at the top of the PR instead of the full output

How do you stop console changes and agent PRs from overwriting each other?


r/Terraform • • 1d ago

Help Wanted Explicitly overridden data source still "unknown" in test

3 Upvotes

I am trying to write some tests, and stumbled upon "Unknown condition value" which I cannot resolve with any overrides. It gives that error in second assert even though both resource and data source are overridden, why? Tried latest 1.17.2

main.tf:

terraform {
  required_providers {
    github = {
      source  = "integrations/github"
      version = "~> 6.0"
    }
  }
}

resource "github_repository" "main" {
  name = "my-test"
}

data "github_repository_file" "existing_file" {
  repository = github_repository.main.name
  file       = "existing_file.yaml"
}

main.tftest.hcl:

mock_provider "github" {
  override_during = plan
}

override_resource {
  override_during = plan
  target = github_repository.main
}

override_data {
  override_during = plan
  target = data.github_repository_file.existing_file
  values = {
    content = "{}"
  }
}

run "release_defaults" {
  command = plan
  assert {
    condition     = github_repository.main.name == "x"
    error_message = "repo"
  }
  assert {
    condition     = data.github_repository_file.existing_file.content == "x"
    error_message = "content"
  }
}

If I switch reference to repository = "my-test" it works, but I need to keep a dependency.


r/Terraform • • 2d ago

Discussion do you split terraform state per environment or per component

28 Upvotes

trying to decide how to carve up our state files. one per environment is easy to follow but each one keeps growing. one per component keeps plans small but then outputs have to be shared between them. what do you use and what made you pick it


r/Terraform • • 3d ago

Discussion Do junior DevOps engineers need to write Terraform from memory?

95 Upvotes

Hey everyone,

I'm learning DevOps and trying to land my first junior role. I've been practicing Linux, Docker, GitHub Actions, CI/CD, GCP, and recently started learning Terraform.

I understand what Terraform does and how the resources work, but I still struggle to write the configurations from scratch without looking at examples or documentation.

For people actually working in DevOps, how much Terraform syntax do you remember? Are juniors expected to write configurations from memory, or is understanding the concepts and being able to work with documentation enough?

I'm trying to figure out what level I should reach before applying for jobs.

Thanks!


r/Terraform • • 2d ago

Discussion The community Neon Terraform provider is now official as neondatabase/neon

Thumbnail reddit.com
5 Upvotes

r/Terraform • • 2d ago

Discussion Do you block every Terraform plan that replaces a database?

0 Upvotes

Say a PR moves an RDS instance into a module. The author expects the database to stay put, but the plan says it’ll be replaced. That’s an easy call in review. What I’m curious about is what happens when the replacement is intentional. Does the same check block it? Who can clear it, and what do they want to see first?


r/Terraform • • 4d ago

Discussion do you keep one state per environment or one per team and why

17 Upvotes

i keep seeing both setups. one state per environment is simple to reason about but gets big and slow. one per team limits the blast radius but then shared stuff like networking gets awkward. curious where people draw the line.


r/Terraform • • 4d ago

Discussion Terraform Provider Development Consulting

8 Upvotes

Does anyone have recommendations for consultants or service providers that specialize in Terraform provider development and maintenance?

Curious what experiences others have had with getting outside help to improve existing providers that have struggled with quality, reliability, and ongoing maintenance.

Would be interested to hear who you’ve worked with and how it went.


r/Terraform • • 6d ago

Discussion How do you get cloud cost fixes into Terraform modules other teams own?

6 Upvotes

I recently fixed about 30 easy cost recommendations by hand in the console and a couple of weeks later one team ran terraform apply on their module and put half of them back. Their lead was fairly pissed that I’d touched their RDS without a PR, which is fair.

The list comes from a cost tool finance bought last year. It’s at 412 open right now and most of it is gp2 volumes, staging RDS sized like prod and a NAT gateway carrying S3 traffic that should be on an endpoint. Our FinOps person files each one as a Jira ticket with the team that owns the module and when I went through them, only 9 of the 412 had ever been picked up. The one for orders-db-stg says est. $212/mo and has been sitting in that team’s backlog for about 8 months.

My manager’s answer is to buy something that makes the changes itself so I sat through two demos last week. One wanted an IAM role that could modify RDS directly, which is how I got here in the first place and the other opened a Jira ticket with a link back to their dashboard.

If you’ve had to resize RDS in modules owned by other teams, who opened the PR and how did you get them to merge it?


r/Terraform • • 6d ago

Discussion Is there a live example of a terraform repo with best practices?

75 Upvotes

I recently joined a new organization as a devops engineer, it's a startup and a small team, the guy before me has vibecoded the infrastructure and it's bad (everything has drifted, hardcoded arms etc).

Our usecase is simple, an ecs cluster, a database, few lambdas and s3. And ofcourse vpc, route53, alb, iam.

Im looking to refactor this and before I make the big decision I want to see real world solutions for infrastructure at this small scale.


r/Terraform • • 6d ago

Discussion Best way to override Terraform provider-level default_tags without modifying the existing CICD framework?

11 Upvotes

Hey there,

We have around 1,200 microservices on AWS deployed using Terraform, with each microservice's Terraform code maintained in a separate repository.

All these repositories use exactly the same AWS provider configuration, which is generated from a common boilerplate. We also have a centralised CI/CD framework built on Azure DevOps that handles deployments for all these microservices.

We now have a requirement to introduce two additional tags across all existing microservices and ensure that any new microservices created through the CI/CD framework automatically inherit these tags.

I'm exploring the following approach:

  • As part of the CI/CD pipeline, read the existing Terraform provider configuration.
  • Dynamically generate a override.tf file containing the additional tags.
  • Use this generated file during the Terraform plan and apply operations.

This would allow us to introduce the tags centrally without having to modify all 1,200 repositories individually. Now my questions are:

  1. Is using a dynamically generated override.tf a good approach for this use case?
  2. Is there a more native or maintainable Terraform approach to achieve this?
  3. Are there any potential issues with using override.tf in a centralised CI/CD pipeline, particularly around provider configuration or resource tagging?

Ideally, I'm looking for a solution that requires minimal changes to the existing repositories while ensuring that all current and future microservices inherit these tags automatically.

Would appreciate any suggestions or recommendations!

Thanks


r/Terraform • • 10d ago

Help Wanted Working with data in terraform

11 Upvotes

I've recently been using terraform to import data from yaml files and manipulate that data into terraform types suitable to be applied via resources.

I don't have much experience manipulating / modifying data in terraform and would like to learn it at a fundamental level. Are there any go to learning resources to assist with this area of terraform?

Mainly I'm interested in learning:

  • What terraform types look like when compared to json / yaml
  • How to visualise terraform types in my head to make data manipulation easier
  • When to use logic in locals vs resources

Any help / direction would be appreciated - thank you.


r/Terraform • • 9d ago

Announcement I built an open-source pre-apply check that stops coding agents from destroying prod resources

0 Upvotes

Disclosure: I'm one of the people building this at Flurit.

Coding agents now run terraform apply as easily as ls, and most guards for that match command text. That breaks once the command is wrapped in base64, eval or a script. We wrote 17 dangerous Terraform scenarios, and the best open-source guard we tried caught 5.

applygate is a hook that runs before every command an agent issues and answers allow, ask or block:

- it parses the command the way the shell would, so wrappers don't hide it

- it reads the saved plan, so an ordinary apply that would delete or replace a resource counts as a destroy

- environments have tiers: prod blocks it, staging asks you, disposable environments let it through

- no model in the decision, so the same plan always gets the same answer and a reason

The same plan check also runs as a GitHub Action, so CI can block or ask on destroyed databases, removed protection or IAM wildcards before apply.

It caught all 17, but they're our own scenarios, so treat that as a demo rather than a coverage claim. Works with Claude Code, Codex and Terragrunt. Apache-2.0, and it starts in observe mode, so it blocks nothing until you turn it on.

https://github.com/Flurit/applygate

How are you handling agents and apply today: plan-only for agents, or something else?


r/Terraform • • 10d ago

Discussion AI-assisted Terraform changes: how do you keep them safe in existing infrastructure?

0 Upvotes

One thing I've been thinking about lately is how AI can help with Terraform without turning infrastructure management into a "let the LLM generate some HCL and hope for the best" exercise.

Generating Terraform configuration from a natural-language prompt is becoming relatively easy. The harder part is dealing with infrastructure that already exists:

Understanding the current state and repository structure.

Detecting and handling infrastructure drift.

Making sure generated changes actually pass terraform validate and terraform plan.

Applying existing architecture and security policies.

Keeping pull requests, reviews and human approval in the loop.

At Inditex Tech, we've been working on this problem with Kumoss, an open-source project that explores a developer-first approach to AI-assisted infrastructure management.

The idea is to use specialised AI agents to help with infrastructure changes, while keeping Terraform/OpenTofu as the source of truth for validation and planning.

A few interesting aspects of the approach:

Changes are generated in a dedicated Git branch.

Terraform/OpenTofu validates the configuration and generates the plan.

Drift is reconciled without blindly reverting changes already present in the environment.

Compliance checks and organisational policies are part of the workflow.

Applying infrastructure changes remains an explicit, authorised human action.

We recently published a technical article explaining the architecture and some of the engineering decisions behind it.

Article: https://medium.com/@InditexTech/managing-ai-assisted-cloud-infrastructure-with-an-open-source-developer-first-workflow-66a005b9b508

Kumoss is released under Apache 2.0.

I'm curious about how other Terraform users approach this:

Would you be comfortable introducing an AI agent into your Terraform workflow if it could generate changes and plans, but couldn't apply anything without human approval?

And more importantly, what would be your biggest concern: state management, drift, security, compliance, or trusting the generated plan?

Interested in hearing how the community sees this evolving.


r/Terraform • • 12d ago

Discussion How do you find the blast radius of a Terraform PR

12 Upvotes

Half our Terraform changes are written by agents now and the diffs look tiny. Last week a 9 line PR changed a variable default in a shared module. The plan for the workspace in the PR was clean. The problem was 4 other workspaces that use the same module, and one of them replaced a load balancer on its next apply.

We have coderabbit on the infra repo and it reviews the HCL itself fine, it caught a security group opened to 0.0.0.0/0 last month. But it only sees the diff, and the actual blast radius here was in workspaces nobody had planned yet.

What we're doing now is ugly: a script that finds every workspace referencing a changed module and runs plan on all of them in CI. It works but it takes 25 minutes on a big change and people have started skipping it.

How are you handling this?

- Do you plan every consumer of a shared module on every PR?

- Pin module versions so a change only lands when each workspace opts in?

- Any tooling that maps "this module change affects these resources" without a full plan everywhere?

Especially interested if you have a lot of agent written infra changes coming in


r/Terraform • • 12d ago

Discussion Thrown to the wolves

15 Upvotes

Where to start...

I am a developer with limited experience managing a handful of VPS(docker networks, nginx reverse-proxy, etc...)

Recently I was contracted out by a company to do some IPsec work using Oracle Cloud as the tunnel between the end app and an auth service. The end app service cannot communicate over an encrypted protocol, so setting up a service inbetween to work with LDAPS.

I have done nothing but down-play my limited expertise. I want to deliver, not just get a paycheck. I made the mistake of diving into an IoT project and setting up an end to end solution from the edge to power bi reports for them a while back with no previous experience. Now they seem to think I can deliver the world to them lol

They just added a full terraform configuration to the list of wants for this project. It's a great idea, but I have never used terraform in my life and they know it. I know OCI has an automatic resource discovery tool to get me started, and Terrarform can automatically pull info from Authentik to recreate those configs but god... should I be this intimidated?

Plan is IF I can get it working to hand them the whole config in a git repository. Is this the right approach?

Tell me its not as bad as it sounds... or that it is and I can advise them that it might be a little over my head for now.


r/Terraform • • 12d ago

Terraform Azure Tutorial for Beginners | Build a VNet and VM from Scratch

Thumbnail youtu.be
0 Upvotes

r/Terraform • • 13d ago

Discussion Anyone else feel like Terraform state is just a rough guess now?

0 Upvotes

Anyone else feel like Terraform state is just a rough guess now because something outside keeps changing resources? i finally traced one to an agent running as the wrong service account in Dome Systems.


r/Terraform • • 13d ago

I built a free tool that shows which repos use a Terraform module before you change it

Post image
0 Upvotes

If you keep shared Terraform modules in their own repos, you’ve probably had this: you change a module, bump the version, and later find out three other repos were using it in ways you didn’t know about.

ripplemap reads the module sources in your repos (git URLs, registry modules, refs) and builds a map of which repo uses which module, at which version. Click a module and it shows every repo a change would affect, down to the file and line.

Example on Cloud Posse’s 163 public module repos: their null-label module is used, directly or through other modules, by 149 of them. Their security-group module is used by 12 repos across 3 different versions.

You can try it on any public GitHub org in your browser, with nothing to install: https://zintrivious.github.io/ripplemap/

For private repos, the CLI runs locally and nothing leaves your machine:

It also works with Azure DevOps, and there’s a GitHub Action that comments on pull requests with the repos a module change may affect.

Free and MIT licensed: https://github.com/Zintrivious/ripplemap

I’d really like to hear where it gets things wrong. Module sources come in a lot of shapes, and I’d rather fix the ones it misses.


r/Terraform • • 15d ago

AWS PR to Add env var inputs to CodePipeline resource in AWS Provider

0 Upvotes

Hopefully this reaches anybody else who is wanting this feature. It's super helpful for the generic commands action.

Please give it a thumbs up if you think it would help you.

If anyone has suggestions on getting these things reviewed please let me know!

https://github.com/hashicorp/terraform-provider-aws/pull/50770


r/Terraform • • 16d ago

Discussion Anyone tried Floci or Vera?

8 Upvotes

Has anyone tried floci or vera to test terraform in emulated environments? Any recommendation, reviews?

I'm looking for a open source emulator to test tf with and emulator. Floci afaik only supports aws but does anyone tried?


r/Terraform • • 15d ago

Tutorial How We Use Terramate for Code Generation to Manage 500+ Terraform Stacks in a Monorepo

0 Upvotes

I wanted to share an article I wrote about structuring a Terraform monorepo with Terramate

https://oponomarov.com/blog/posts/structuring-a-terraform-monorepo-with-terramate/

I feel like Terramate is still relatively unknown, and even fewer people realize that you can use it purely for code generation, without adopting its other features.

That’s exactly how we use it to manage and template 500+ distinct Terraform stacks across different regions, environments, and other dimensions.

This is the first post in a six-part series where I share our experience using Terramate to manage a large-scale Terraform monorepo in practice.

If you’ve ever struggled with organizing hundreds of Terraform stacks, managing configuration across multiple dimensions, or figuring out how to keep everything maintainable as your infrastructure grows, you might find some useful ideas in the series.

Hopefully, it helps someone facing similar challenges.


r/Terraform • • 17d ago

Discussion terraform-provider-system (agentless SSH: files, users, packages) has been quiet since May 2024. Who still uses it? I have a changeset I'd like to propose upstream.

0 Upvotes

neuspaces/terraform-provider-system manages files, folders, users, groups, packages and services on a remote host over SSH, with no agent — the escape hatch for pets in an otherwise immutable setup.

Last commit: 31 May 2024. v0.5.0 is still the latest release. 45 stars, 11 forks, 4 open issues — and the registry reports ~523k downloads, so a fair number of terraform init runs are still pulling it.

I use it for a handful of servers, and I've spent a sprint on a changeset against it. Before I open an issue upstream and send pull requests, I'd like a reality check from people who actually run this thing. I am not publishing a competing provider. The work sits in a plain GitHub fork, on a branch that keeps the module path and the provider address as they are, with no trace of a fork in the tree — it is shaped to be merged upstream, not released beside it.

The change I'm least sure about

Refusing to connect when host_key is not set. Today the provider silently falls back to ssh.InsecureIgnoreHostKey() if you leave host_key out. Every such connection is open to MITM and nothing in the plan says so. My branch makes ConfigureProvider fail unless you set host_key (also proxy.ssh.host_key behind a bastion) or opt out with a new insecure_ignore_host_key = true, which warns on every run and names the block whose key is missing. ssh-keyscan -t ed25519 <host> gets you the value.

That breaks every existing configuration on upgrade, which is why I'm asking instead of just sending it:

  • Hard failure with an explicit opt-out, or warn first and fail a release later?
  • Does anyone actually set host_key today, or is everyone on the implicit insecure path?
  • If you go through a bastion, does requiring proxy.ssh.host_key hurt?

The rest of the changeset

  • overwrite for system_file and system_folder — adopt an existing path on create instead of failing. This is upstream PR #11 by raynigon (issue #4), rebased and finished: symlinks still fail, even when they point at an entry of the right kind, so nothing is ever written through a link.
  • create_home for system_user — upstream PR #10 by ii1818nn (issue #9). The default keeps passing --no-create-home, so behaviour doesn't depend on CREATE_HOME in /etc/login.defs.
  • system_file.source is no longer ForceNew: changed content is rewritten in place instead of destroy + create. Tell me if you relied on the replacement.
  • Removing source from a resource no longer recreates the file empty and throws the remote content away.
  • Fixed: switching between content, content_sensitive and source in one apply wrote nothing at all.
  • Dependencies to Go 1.27 and SDKv2 2.40.1. govulncheck on v0.5.0 reports 28 reachable vulnerabilities, nine of them in the SSH path; the branch reports none, and the check gates CI.
  • CI on GitHub-hosted runners: Terraform 1.6 / 1.10 / 1.11 / 1.14 / 1.17 and OpenTofu 1.6 / current, acceptance against Debian, Alpine and Fedora containers built inside the workflow, so a pull request from a fork needs no pre-published images. The README claimed 0.12+; the real floor is 1.6. Protocol stays 5, existing state is unaffected.
  • An acceptance test for the thing I care about most: a state file written by the released v0.5.0 binary must produce an empty plan with the new code, on all three target systems, across the whole CLI matrix, with no change to required_providers.

The full matrix is green — three target systems by seven CLIs, twenty-one acceptance runs, compatibility test included.

One thing the matrix found, and I'd like an opinion on it

The provider offers a password as both the password and the keyboard-interactive method. Against an sshd built without PAM — the Alpine target, for one — the server answers the keyboard-interactive request with USERAUTH_FAILURE and no challenge at all, and current golang.org/x/crypto/ssh aborts the authentication loop with unexpected message type 51 (expected 60) rather than reporting the exhausted method list. It behaves that way since it started to require a challenge before it accepts the result of a method; the version v0.5.0 pins has no such requirement.

The practical effect: on those servers, a mistyped password now produces a protocol error instead of "unable to authenticate". Authentication itself is unaffected — the correct password connects, the wrong one is refused — but the message is useless to a human. Is this worth translating in the provider, or do you read raw x/crypto errors and shrug?

Later, and only if there's interest: moving the file, folder and link resources to the plugin framework behind terraform-plugin-mux, which costs exactly one in-place update of source on the first plan (the framework can't keep etag=… in that attribute), plus content_wo for write-only content on Terraform 1.11+.

What I'm asking

  1. Do you use this provider? On what, and how many hosts?
  2. What keeps you on 0.5.0, or what made you drop it?
  3. Which of the four open issues actually bites you?
  4. Has anyone reached the maintainers recently? I'd rather contribute than fork, and the repo isn't archived — but eighteen months on a vulnerable SSH stack is why I started.

Branch, if you'd rather read code than prose: https://github.com/metronom72/terraform-provider-system/pull/1