Skip to content

Commit aaf8333

Browse files
aknyshautofix-ci[bot]claude
authored
Update Atmos YAML functions docs (#1826)
* update docs * update docs * update docs * [autofix.ci] apply automated fixes * docs: Address CodeRabbit nitpick suggestions - Clarify that !terraform.state returns null when state file doesn't exist - Enhance YQ expression comments in store.get.mdx to be more specific 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * address comments * address comments --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Claude <[email protected]>
1 parent af58038 commit aaf8333

File tree

9 files changed

+170
-10
lines changed

9 files changed

+170
-10
lines changed

website/docs/functions/index.mdx

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -92,11 +92,11 @@ If you must use template functions:
9292

9393
If you have existing template functions, consider migrating to YAML functions:
9494

95-
| Instead of (Template) | Use (YAML) |
96-
|----------------------|------------|
97-
| `{{ exec "command" }}` | `!exec command` |
98-
| `{{ env "VAR" }}` | `!env VAR` |
99-
| `{{ toJson .values }}` | Native YAML structures |
95+
| Instead of (Template) | Use (YAML) |
96+
|-------------------------------|-------------------------------------------|
97+
| `{{ exec "command" }}` | `!exec command` |
98+
| `{{ env "VAR" }}` | `!env VAR` |
99+
| `{{ toJson .values }}` | Native YAML structures |
100100
| Reading outputs via templates | `!terraform.output` or `!terraform.state` |
101101

102102
## Performance Considerations

website/docs/functions/yaml/env.mdx

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,11 @@ The `env:` section follows the standard Atmos merge priority:
8080
This means values defined at the component level will override values from globals.
8181
:::
8282

83+
:::note Type-Aware Merging
84+
Atmos supports type-aware merging of YAML functions and concrete values, allowing them to coexist in the inheritance chain without type conflicts.
85+
See the full explanation: [YAML Function Merging](/reference/yaml-function-merging)
86+
:::
87+
8388
## Limitations
8489

8590
**Single-Pass Processing**: YAML functions are processed in a single pass. This means `!env` cannot reference environment variables that are defined by other YAML functions in the same component section.

website/docs/functions/yaml/exec.mdx

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,3 +67,8 @@ You can use [Atmos Stack Manifest Templating](/core-concepts/stacks/templates) i
6767
Atmos processes the templates first, and then executes the `!exec` function, allowing you to provide the parameters to
6868
the function dynamically.
6969
:::
70+
71+
:::note Type-Aware Merging
72+
Atmos supports type-aware merging of YAML functions and concrete values, allowing them to coexist in the inheritance chain without type conflicts.
73+
See the full explanation: [YAML Function Merging](/reference/yaml-function-merging)
74+
:::

website/docs/functions/yaml/store.get.mdx

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -78,20 +78,25 @@ If you need to retrieve values that follow the Atmos stack/component/key pattern
7878
[`!store`](/functions/yaml/store) function instead, as it provides better integration with Atmos component dependencies.
7979
:::
8080

81+
:::note Type-Aware Merging
82+
Atmos supports type-aware merging of YAML functions and concrete values, allowing them to coexist in the inheritance chain without type conflicts.
83+
See the full explanation: [YAML Function Merging](/reference/yaml-function-merging)
84+
:::
85+
8186
### Using YQ Expressions
8287

8388
You can use [YQ](https://mikefarah.gitbook.io/yq) expressions to extract specific values
8489
from complex data structures:
8590

8691
<File title="stack.yaml">
8792
```yaml
88-
# Extract a field from a JSON object
93+
# Extract database host from app-config in Redis
8994
database_host: !store.get redis app-config | query .database.host
9095

91-
# Get the first item from an array
96+
# Get the first availability zone from Azure Key Vault array
9297
primary_zone: !store.get azure-keyvault availability-zones | query .[0]
9398

94-
# Extract nested values with a default
99+
# Extract API key from SSM config with empty object fallback
95100
api_key: !store.get ssm /external/config | default "{}" | query .api.key
96101
```
97102
</File>

website/docs/functions/yaml/store.mdx

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,11 @@ Atmos processes the templates first, and then executes the `!store` function, al
7373
the function dynamically.
7474
:::
7575

76+
:::note Type-Aware Merging
77+
Atmos supports type-aware merging of YAML functions and concrete values, allowing them to coexist in the inheritance chain without type conflicts.
78+
See the full explanation: [YAML Function Merging](/reference/yaml-function-merging)
79+
:::
80+
7681
## Using YQ Expressions to retrieve individual values
7782

7883
To retrieve individual values from complex types such as maps and lists, or do any kind of filtering or querying,

website/docs/functions/yaml/template.mdx

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,11 @@ YAML function. It produces the same results, correctly handles the complex types
139139

140140
:::
141141

142+
:::note Type-Aware Merging
143+
Atmos supports type-aware merging of YAML functions and concrete values, allowing them to coexist in the inheritance chain without type conflicts.
144+
See the full explanation: [YAML Function Merging](/reference/yaml-function-merging)
145+
:::
146+
142147
## Advanced Examples
143148

144149
The `!template` Atmos YAML function can be used to make your stack configuration DRY and reusable.

website/docs/functions/yaml/terraform.output.mdx

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,11 @@ Atmos processes the templates first, and then executes the `!terraform.output` f
5454
the function dynamically.
5555
:::
5656

57+
:::note Type-Aware Merging
58+
Atmos supports type-aware merging of YAML functions and concrete values, allowing them to coexist in the inheritance chain without type conflicts.
59+
See the full explanation: [YAML Function Merging](/reference/yaml-function-merging)
60+
:::
61+
5762
## `!terraform.output` Function Execution Flow
5863

5964
When processing the `!terraform.output` YAML function for a component in a stack, Atmos executes the following steps:
@@ -147,6 +152,9 @@ endpoint: !terraform.output services '.endpoints["my-service"]["production"].url
147152
148153
# Combine with stack templating
149154
token: !terraform.output identity {{ .stack }} '.tokens["github-actions"].value'
155+
156+
# Escape single quotes by doubling them when needed inside the expression
157+
app_name: !terraform.output config '.apps["app''s-name"].display_name'
150158
```
151159

152160
**Quote escaping rules**

website/docs/functions/yaml/terraform.state.mdx

Lines changed: 41 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,11 @@ Atmos processes the templates first, and then executes the `!terraform.state` fu
6767
the function dynamically.
6868
:::
6969

70+
:::note Type-Aware Merging
71+
Atmos supports type-aware merging of YAML functions and concrete values, allowing them to coexist in the inheritance chain without type conflicts.
72+
See the full explanation: [YAML Function Merging](/reference/yaml-function-merging)
73+
:::
74+
7075
## `!terraform.state` Function Execution Flow
7176

7277
When processing the `!terraform.state` YAML function for a component in a stack, Atmos executes the following steps:
@@ -128,7 +133,40 @@ For more details, review the following docs:
128133
129134
## Handling YQ Expressions with Bracket Notation and Quotes
130135
131-
`!terraform.state` shares the same parser as `!terraform.output`, so the quoting rules are identical. When you need double quotes inside a YQ expression (for example, to reference `["github-dependabot"]`), wrap the entire expression in single quotes. Refer to the [!terraform.output guidance](/functions/yaml/terraform.output#handling-yq-expressions-with-bracket-notation-and-quotes) for detailed examples and escape rules.
136+
When you access map keys that contain special characters (such as hyphens) by using YQ bracket notation, you must wrap the key in double quotes like `["github-dependabot"]`. This often clashes with the surrounding quotes that protect the entire expression in YAML.
137+
138+
To avoid conflicting quotes, wrap the entire YQ expression in single quotes while keeping the double quotes inside the brackets:
139+
140+
```yaml
141+
# Use single quotes around the expression to allow double quotes inside brackets
142+
access_key_id: !terraform.state security '.outputs.users["github-dependabot"].access_key_id'
143+
```
144+
145+
Additional examples:
146+
147+
```yaml
148+
# Access map keys with special characters
149+
api_key: !terraform.state config '.outputs.api_keys["service-account-1"]'
150+
151+
# Access nested maps with special characters in multiple levels
152+
endpoint: !terraform.state services '.outputs.endpoints["my-service"]["production"]'
153+
154+
# Combine with stack templating
155+
token: !terraform.state identity {{ .stack }} '.outputs.tokens["github-actions"]'
156+
157+
# Escape single quotes by doubling them when needed inside the expression
158+
app_name: !terraform.state config '.outputs.apps["app''s-name"].display_name'
159+
```
160+
161+
**Quote escaping rules**
162+
163+
- Wrap the entire YQ expression in single quotes when it contains double quotes.
164+
- Use double quotes inside brackets for string keys in YQ expressions.
165+
- If you need single quotes inside the expression, escape them by doubling: `''`.
166+
167+
:::tip
168+
This quoting pattern works for every Atmos YAML function that accepts YQ expressions, including [!terraform.output](/functions/yaml/terraform.output) and [!include](/functions/yaml/include).
169+
:::
132170

133171
## Using YQ Expressions to provide a default value
134172

@@ -483,4 +521,5 @@ vars:
483521

484522
- Be mindful of disaster recovery (DR) implications when using it across regions.
485523

486-
- Consider cold-start scenarios: if the dependent component has not yet been provisioned, `terraform output` will fail.
524+
- Consider cold-start scenarios: if the referenced component's state file doesn't exist (e.g., not yet provisioned), `!terraform.state` returns `null`.
525+
Use the YQ `//` operator to provide default values for components that may not be provisioned yet (see [Using YQ Expressions to provide a default value](#using-yq-expressions-to-provide-a-default-value)).
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
---
2+
title: Type-Aware Merging of YAML Functions
3+
sidebar_position: 4
4+
sidebar_label: YAML Function Merging
5+
description: How Atmos handles merging YAML functions with concrete values in the inheritance chain
6+
---
7+
8+
import Intro from '@site/src/components/Intro'
9+
10+
<Intro>
11+
Atmos supports type-aware merging of YAML functions and concrete values, allowing them to coexist in the inheritance chain without type conflicts.
12+
</Intro>
13+
14+
## The Problem: Type Conflicts During Merge
15+
16+
When merging stack configurations, Atmos needs to handle both concrete values and YAML function references. Without special handling, the merge process would encounter type mismatches:
17+
18+
```yaml
19+
# Base catalog (catalog/app/defaults.yaml)
20+
components:
21+
terraform:
22+
my-app:
23+
vars:
24+
database_url: "postgresql://localhost:5432/mydb" # Concrete string value
25+
26+
# Environment override (stacks/prod/app.yaml)
27+
components:
28+
terraform:
29+
my-app:
30+
vars:
31+
database_url: !env DATABASE_URL # YAML function reference
32+
```
33+
34+
The YAML function `!env DATABASE_URL` is a different type than the string `"postgresql://localhost:5432/mydb"`. Without special handling, merging these would cause a type conflict error.
35+
36+
## The Solution: Type-Aware Merging
37+
38+
Atmos handles this gracefully through **type-aware merging**:
39+
40+
1. YAML functions are recognized and wrapped in a special type during parsing
41+
2. During merge, this allows YAML functions to replace concrete values (and vice versa) without type errors
42+
3. YAML functions are evaluated as the final step in stack processing, as they have always been
43+
44+
This approach ensures that YAML functions can override static values throughout the inheritance chain.
45+
46+
## Supported Functions
47+
48+
The following YAML functions support type-aware merging:
49+
50+
- [`!env`](/functions/yaml/env) - Environment variables
51+
- [`!exec`](/functions/yaml/exec) - External command execution
52+
- [`!store`](/functions/yaml/store) - Store value retrieval
53+
- [`!store.get`](/functions/yaml/store.get) - Store value retrieval (alias)
54+
- [`!template`](/functions/yaml/template) - Template rendering
55+
- [`!terraform.output`](/functions/yaml/terraform.output) - Terraform output values
56+
- [`!terraform.state`](/functions/yaml/terraform.state) - Terraform state values
57+
58+
## Benefits
59+
60+
- **Flexible Configuration Patterns** - Mix static values and YAML functions across inheritance layers without type conflicts
61+
- **Gradual Migration** - Migrate from static to dynamic configurations incrementally
62+
- **Team Collaboration** - Different teams can use different approaches (static vs. templated) in their layers
63+
- **Multi-Environment Support** - Use static values in dev/staging and YAML functions in production
64+
65+
## Example: Mixed Static and Dynamic Values
66+
67+
```yaml
68+
# catalog/base.yaml - Base component with static defaults
69+
components:
70+
terraform:
71+
my-service:
72+
vars:
73+
log_level: "info"
74+
api_endpoint: "https://api.dev.example.com"
75+
76+
# stacks/prod.yaml - Production uses dynamic values
77+
import:
78+
- catalog/base
79+
80+
components:
81+
terraform:
82+
my-service:
83+
vars:
84+
log_level: !env LOG_LEVEL
85+
api_endpoint: !terraform.output api-gateway endpoint
86+
```
87+
88+
In this example, the production stack overrides static defaults with YAML functions. Type-aware merging allows these different value types to coexist without errors.

0 commit comments

Comments
 (0)