Not everything your configuration needs is created by that configuration. The shared reverse proxy network was created by another team. The latest approved machine image is published by a build pipeline. Your AWS account ID depends on which credentials run the plan. Data sources let Terraform look these things up at plan time instead of you pasting IDs into code.
Resource versus data source
resource | data | |
|---|---|---|
| What Terraform does | Creates, updates, destroys | Only reads |
| Appears in plan as | +, ~, - | <= (read) |
Destroyed by terraform destroy | Yes | Never: nothing to destroy |
| Referenced as | docker_network.orbit.id | data.docker_network.proxy.id |
The syntax is the same block shape with data instead of resource. The arguments are filters (what to look for) and the attributes are results.
Reading something another team owns
On the shared staging host, a reverse proxy runs on a network called edge, created by the platform team’s own configuration. Orbit’s web container must join it, but Orbit must never modify or delete it:
data "docker_network" "edge" {
name = "edge"
}
resource "docker_container" "web" {
name = "orbit-${var.environment}-web"
image = docker_image.web.image_id
networks_advanced {
name = data.docker_network.edge.name
}
}
output "edge_subnet" {
value = data.docker_network.edge.ipam_config
}
If the edge network does not exist, plan fails with a clear “not found” error, which is what you want: a missing dependency should stop the deploy, not create a surprise copy.
Pinning an image by digest
Tags like nginx:1.27-alpine are moved by their publisher when a patch is released. The registry data source turns a tag into the immutable digest it points to right now:
data "docker_registry_image" "web" {
name = "nginx:1.27-alpine"
}
resource "docker_image" "web" {
name = data.docker_registry_image.web.name
pull_triggers = [data.docker_registry_image.web.sha256_digest]
}
When the publisher moves the tag, the digest changes, pull_triggers changes, and the next plan shows the image (and the container using it) being replaced. You see the update in a plan instead of discovering it after a restart.
Lookups you will use in the cloud
Lesson 12 uses AWS. These four data sources appear in almost every AWS configuration:
data "aws_caller_identity" "current" {}
data "aws_availability_zones" "available" {
state = "available"
}
data "aws_ssm_parameter" "al2023_ami" {
name = "/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64"
}
data "aws_iam_policy_document" "read_uploads" {
statement {
actions = ["s3:GetObject"]
resources = ["arn:aws:s3:::orbit-uploads-${data.aws_caller_identity.current.account_id}/*"]
}
}
aws_caller_identitygives the account ID of whoever runs the plan, so bucket names and ARNs are never hard-coded per account.aws_availability_zoneslists the zones of the region instead of assuminga,bandcexist.- The SSM parameter is the official, always-current Amazon Linux image ID. It is simpler and less error-prone than searching AMIs by name with
aws_ami. aws_iam_policy_documentbuilds IAM JSON from HCL: no hand-written JSON, no quoting mistakes.
Reading another configuration’s outputs
Orbit’s network lives in its own configuration and state, and the application configuration needs the subnet IDs. The terraform_remote_state data source reads another state’s outputs:
data "terraform_remote_state" "network" {
backend = "s3"
config = {
bucket = "orbit-tfstate-123456789012"
key = "orbit/network/terraform.tfstate"
region = "eu-west-1"
}
}
locals {
private_subnet_ids = data.terraform_remote_state.network.outputs.private_subnet_ids
}
It works, but it needs read access to the whole network state, secrets included. Two alternatives are often better: publish the values to a parameter store and read them with a normal data source, or look the subnets up directly with aws_subnets and a tag filter.
Secrets: use ephemeral resources, not data sources
A data source’s results are saved in state. If you read a database password with data "aws_secretsmanager_secret_version", the password ends up in the state file. Terraform 1.10 added ephemeral resources, which fetch a value for the duration of a run and never write it to state or plan:
ephemeral "aws_secretsmanager_secret_version" "db" {
secret_id = "orbit/staging/db"
}
provider "postgresql" {
host = aws_db_instance.orbit.address
username = "orbit_admin"
password = jsondecode(ephemeral.aws_secretsmanager_secret_version.db.secret_string)["password"]
}
Ephemeral values can flow into provider blocks, other ephemeral values and write-only arguments. Lesson 15 puts the full secret story together.
Bringing existing objects under Terraform: import blocks
Data sources read things you do not want to manage. When you do want Terraform to take over something created by hand, import it. Since Terraform 1.5 that is a block in code, reviewed like any other change:
import {
to = docker_volume.uploads
id = "orbit-staging-uploads"
}
You can even let Terraform write the resource block for you:
$ terraform plan -generate-config-out=generated.tf
# docker_volume.uploads will be imported
Plan: 1 to import, 0 to add, 0 to change, 0 to destroy.
$ terraform apply
Review generated.tf, tidy it up (remove computed attributes, replace hard-coded values with references) and move it into your normal files. After the apply, the import block can be deleted. Import blocks also accept for_each (1.7+) for importing many objects at once, and since 1.16 they work inside modules.
Finding what to import: terraform query
Importing needs an ID for every object, which is painful when you are adopting a whole account built by hand. Terraform 1.14 added terraform query, which lists existing infrastructure through providers that support it. You write list blocks in files ending in .tfquery.hcl:
# discover.tfquery.hcl
list "aws_instance" "all" {
provider = aws
}
$ terraform query
$ terraform query -generate-config-out=to-import/instances.tf
The first command prints what was found. The second writes import and resource blocks for every result into a new file, ready to review. Provider-specific filters (by tag, by name) go in a config { } block inside list; check the provider documentation for which resource types support listing.
When data is read
Terraform reads data sources during plan whenever their arguments are known. If an argument depends on a resource that does not exist yet, the read is deferred to apply, and every attribute that depends on it shows (known after apply). That can make a plan much less informative, so avoid feeding data sources with values from resources created in the same run when a plain reference would do.
Do not add depends_on to data sources “just in case”: it has the same effect and pushes the read to apply time on every run.
Quick reference
| Task | Syntax or command |
|---|---|
| Read an existing object | data "docker_network" "edge" { name = "edge" } |
| Reference it | data.docker_network.edge.id |
| Current AWS account ID | data.aws_caller_identity.current.account_id |
| Another state’s outputs | data.terraform_remote_state.network.outputs.<name> |
| Fetch a secret without storing it (1.10) | ephemeral "<type>" "<name>" { ... } |
| Adopt an existing object (1.5) | import { to = ..., id = "..." } |
| Generate config for imports | terraform plan -generate-config-out=generated.tf |
| List existing infrastructure (1.14) | terraform query |
| Generate imports from a query | terraform query -generate-config-out=to-import/file.tf |
Exercise
Orbit’s containers must join the platform team’s existing edge network, and Orbit must never delete it. How should Orbit’s code refer to it?
Show answer
Answer: A. A data source only reads, so Orbit can use the network without owning it. A resource or import would make Orbit’s terraform destroy delete the shared network.
Exercise
You read a database password with a data source and it works. Why is an ephemeral resource the better choice?
Show answer
Answer: C. Everything a data source returns is saved in state. Ephemeral resources (1.10+) keep the secret in memory only for the run.
Key takeaways
- Data sources read; resources manage. A data source is never destroyed.
- Look up IDs, accounts, zones and images instead of hard-coding them.
- Pin container images by digest with
docker_registry_imageandpull_triggers. terraform_remote_stateworks but exposes the whole state; prefer a parameter store or direct lookups.- Fetch secrets with ephemeral resources so they never reach state.
- Adopt existing objects with
importblocks and generated config; find them withterraform query(1.14).
FAQ
Is terraform import (the command) still used?
It still works, but the import block is better: it shows up in plan, can be reviewed in a pull request, can generate configuration and can import many objects with for_each. Use the command only for quick one-offs.
Why does my plan say a data source will be read during apply?
One of its arguments depends on something not known until apply, often a resource created in the same run or a depends_on. Pass known values where you can so the read happens at plan time.
Can a data source fail the plan if nothing matches?
Most do, with a “not found” or “no matching” error. A few list-style data sources (such as aws_subnets) return an empty list instead; add a postcondition if an empty result should be an error.
Do data sources cost money?
They only make read API calls. Very large configurations with hundreds of lookups can hit API rate limits, which is another reason to keep configurations reasonably small.