Skip to content

Commit 1dbb7c4

Browse files
verygoodsoftwarenotvirusverygoodsoftwarenotvirus
authored
add post (#20)
1 parent e47420b commit 1dbb7c4

File tree

1 file changed

+281
-0
lines changed

1 file changed

+281
-0
lines changed

‎content/posts/env-var-tooling.md‎

Lines changed: 281 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,281 @@
1+
+++
2+
title = "The Gift of Good Tooling"
3+
date = "2025-01-01T11:34:37-06:00"
4+
author = "verygoodsoftwarenotvirus"
5+
cover = ""
6+
tags = []
7+
keywords = []
8+
description = ""
9+
showFullContent = false
10+
readingTime = true
11+
+++
12+
13+
Recently I joined a new team at work, and one my favorite parts of joining a new team is seeing what tools are and aren’t in use, and most importantly, why or why not. I don’t think I’ve ever joined a team that didn’t introduce me to a new library or utility, most good, some bad.
14+
15+
In today’s case, the tool I want to talk about is caarlos0’s [env](https://github.com/caarlos0/env). It’s a library that effectively allows you to map environment variables to the values of individual structs. So if I have a config struct like:
16+
17+
```go
18+
type Config struct {
19+
Debug bool `env:"DEBUG"`
20+
}
21+
```
22+
23+
At runtime, if `Debug` is set to `false`, but the environment variable `DEBUG` is set to `true`, `config.Debug` will be `true` after I use the `env` library to load this struct.
24+
25+
I understood how it all worked, but didn’t understand how valuable it was until we had to diagnose a bug in production at work. We needed to disable something in production and rather than having to create a PR and go through a bunch of CI/CD rigamarole, we edited the pod’s YAML in k9s and confirmed the issue. I felt like a god and I wasn’t responsible for any of it.
26+
27+
My curse and blessing is that I’m the kind of person who cannot experience a great tooling experience and not immediately find a nail for that hammer. I maintain a constantly-morphing side project almost *because* it frequently gives me that nail.
28+
29+
One hangup I had about using it in my side project is documenting the consequent flags. `env` allows you to set a prefix for nested values, and the config I use in my side project is very nested. To adapt our earlier example:
30+
31+
```go
32+
type DatabaseConfig struct {
33+
Debug bool `env:"DEBUG"`
34+
}
35+
36+
type ServiceConfig struct {
37+
Debug bool `env:"DEBUG"`
38+
Database DatabaseConfig `envPrefix:"DATABASE_"`
39+
}
40+
```
41+
42+
This would allow you to set `svcCfg.Database.Debug` by setting the environment variable `DATABASE_DEBUG`. That’s all well and good, easy enough to suss out, but it’s also precisely the sort of mental toil I’m willing to spend more time than I could ever potentially save trying to avoid.
43+
44+
So I (with some help from ChatGPT) wrote some code to parse the AST and produce a library of string constants that document their responsibility:
45+
46+
```go
47+
package main
48+
49+
import (
50+
"fmt"
51+
"go/ast"
52+
"go/parser"
53+
"go/token"
54+
"log"
55+
"os"
56+
"path/filepath"
57+
"slices"
58+
"strings"
59+
60+
"github.com/<side_project>/internal/config"
61+
62+
"github.com/codemodus/kace"
63+
)
64+
65+
func main() {
66+
dir, err := os.Getwd()
67+
if err != nil {
68+
log.Fatal(err)
69+
}
70+
71+
structs := parseGoFiles(dir)
72+
73+
outputLines := []string{}
74+
if mainAST, found := structs["config.APIServiceConfig"]; found {
75+
for envVar, fieldPath := range extractEnvVars(mainAST, structs, "main", "", "") {
76+
outputLines = append(outputLines, fmt.Sprintf(` // %sEnvVarKey is the environment variable name to set in order to override `+"`"+`config%s`+"`"+`.
77+
%sEnvVarKey = "%s%s"
78+
79+
`, kace.Pascal(envVar), fieldPath, kace.Pascal(envVar), config.EnvVarPrefix, envVar))
80+
}
81+
}
82+
83+
slices.Sort(outputLines)
84+
85+
out := fmt.Sprintf(`package envvars
86+
87+
/*
88+
This file contains a reference of all valid service environment variables.
89+
*/
90+
91+
const (
92+
%s
93+
)
94+
`, strings.Join(outputLines, ""))
95+
96+
if err = os.WriteFile(filepath.Join(dir, "internal", "config", "envvars", "env_vars.go"), []byte(out), 0o0644); err != nil {
97+
log.Fatal(err)
98+
}
99+
}
100+
101+
// parseGoFiles parses all Go files in the given directory and returns a map of struct names to their AST nodes.
102+
func parseGoFiles(dir string) map[string]*ast.TypeSpec {
103+
structs := make(map[string]*ast.TypeSpec)
104+
105+
err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
106+
if err != nil {
107+
return err
108+
}
109+
110+
if info.IsDir() || !strings.HasSuffix(info.Name(), ".go") {
111+
return nil
112+
}
113+
114+
if strings.Contains(path, "vendor") {
115+
return filepath.SkipDir
116+
}
117+
118+
node, err := parser.ParseFile(token.NewFileSet(), path, nil, parser.AllErrors)
119+
if err != nil {
120+
fmt.Printf("Error parsing file %s: %v\n", path, err)
121+
return nil
122+
}
123+
124+
for _, decl := range node.Decls {
125+
genDecl, isGenDecl := decl.(*ast.GenDecl)
126+
if !isGenDecl {
127+
continue
128+
}
129+
130+
for _, spec := range genDecl.Specs {
131+
typeSpec, isTypeSpec := spec.(*ast.TypeSpec)
132+
if !isTypeSpec {
133+
continue
134+
}
135+
136+
if _, ok = typeSpec.Type.(*ast.StructType); ok {
137+
key := fmt.Sprintf("%s.%s", node.Name.Name, typeSpec.Name.Name)
138+
structs[key] = typeSpec
139+
}
140+
}
141+
}
142+
return nil
143+
})
144+
145+
if err != nil {
146+
fmt.Printf("Error walking directory: %v\n", err)
147+
}
148+
149+
return structs
150+
}
151+
152+
// getTagValue extracts the value of a specific tag from a struct field tag.
153+
func getTagValue(tag, key string) string {
154+
tags := strings.Split(tag, " ")
155+
for _, t := range tags {
156+
parts := strings.SplitN(t, ":", 2)
157+
if len(parts) == 2 && parts[0] == key {
158+
return strings.Trim(parts[1], "\"")
159+
}
160+
}
161+
return ""
162+
}
163+
164+
// handleIdent handles extracting info from an *ast.Ident node.
165+
func handleIdent(structs map[string]*ast.TypeSpec, fieldType *ast.Ident, envVars map[string]string, currentPackage, prefixValue, fieldNamePrefix, fieldName string) {
166+
for key, nestedStruct := range structs {
167+
keyParts := strings.Split(key, ".")
168+
if len(keyParts) == 2 && keyParts[1] == fieldType.Name {
169+
if keyParts[0] == currentPackage || currentPackage == "main" {
170+
for k, v := range extractEnvVars(nestedStruct, structs, keyParts[0], prefixValue, fmt.Sprintf("%s.%s", fieldNamePrefix, fieldName)) {
171+
envVars[k] = v
172+
}
173+
}
174+
}
175+
}
176+
}
177+
178+
// handleSelectorExpr handles extracting info from an *ast.SelectorExpr node.
179+
func handleSelectorExpr(structs map[string]*ast.TypeSpec, fieldType *ast.SelectorExpr, envVars map[string]string, prefixValue, fieldNamePrefix, fieldName string) {
180+
if pkgIdent, isIdentifier := fieldType.X.(*ast.Ident); isIdentifier {
181+
pkgName := pkgIdent.Name
182+
183+
fullName := fmt.Sprintf("%s.%s", pkgName, fieldType.Sel.Name)
184+
if nestedStruct, found := structs[fullName]; found {
185+
for k, v := range extractEnvVars(nestedStruct, structs, pkgName, prefixValue, fmt.Sprintf("%s.%s", fieldNamePrefix, fieldName)) {
186+
envVars[k] = v
187+
}
188+
}
189+
}
190+
}
191+
192+
// extractEnvVars traverses a struct definition and collects environment variables, resolving nested structs.
193+
func extractEnvVars(typeSpec *ast.TypeSpec, structs map[string]*ast.TypeSpec, currentPackage, envVarPrefix, fieldNamePrefix string) map[string]string {
194+
envVars := map[string]string{}
195+
196+
structType, ok := typeSpec.Type.(*ast.StructType)
197+
if !ok {
198+
return envVars
199+
}
200+
201+
for _, field := range structType.Fields.List {
202+
if field.Tag == nil {
203+
continue
204+
}
205+
206+
tag := strings.Trim(field.Tag.Value, "`")
207+
if tag == `json:"-"` || tag == "" {
208+
continue
209+
}
210+
211+
fn := field.Names[0].Name
212+
213+
if envValue := getTagValue(tag, "env"); envValue != "" {
214+
if envVarPrefix != "" {
215+
envValue = envVarPrefix + envValue
216+
}
217+
218+
if fieldNamePrefix == "" {
219+
envVars[envValue] = fn
220+
} else {
221+
envVars[envValue] = fmt.Sprintf("%s.%s", fieldNamePrefix, fn)
222+
}
223+
}
224+
225+
if prefixValue := getTagValue(tag, "envPrefix"); prefixValue != "" {
226+
if envVarPrefix != "" {
227+
prefixValue = envVarPrefix + prefixValue
228+
}
229+
230+
switch fieldType := field.Type.(type) {
231+
case *ast.Ident:
232+
handleIdent(structs, fieldType, envVars, currentPackage, prefixValue, fieldNamePrefix, fn)
233+
case *ast.SelectorExpr:
234+
handleSelectorExpr(structs, fieldType, envVars, prefixValue, fieldNamePrefix, fn)
235+
case *ast.StarExpr:
236+
switch ft := fieldType.X.(type) {
237+
case *ast.Ident:
238+
handleIdent(structs, ft, envVars, currentPackage, prefixValue, fieldNamePrefix, fn)
239+
case *ast.SelectorExpr:
240+
handleSelectorExpr(structs, ft, envVars, prefixValue, fieldNamePrefix, fn)
241+
}
242+
}
243+
}
244+
}
245+
246+
return envVars
247+
}
248+
249+
```
250+
251+
The only code here specific to my service is the import of a string constant `config.EnvVarPrefix`, which we’ll say for the sake of example is `SIDE_PROJECT_`, and the name of the config struct the code looks for (`config.APIServiceConfig`). Everything else *should* produce a file that looks something like this:
252+
253+
```go
254+
package envvars
255+
256+
/*
257+
This file contains a reference of all valid service environment variables.
258+
*/
259+
260+
const (
261+
// AnalyticsCircuitBreakerErrorRateEnvVarKey is the environment variable name to set in order to override `config.Analytics.CircuitBreakerConfig.ErrorRate`.
262+
AnalyticsCircuitBreakerErrorRateEnvVarKey = "SIDE_PROJECT_ANALYTICS_CIRCUIT_BREAKER_ERROR_RATE"
263+
264+
// AnalyticsCircuitBreakerMinimumSampleThresholdEnvVarKey is the environment variable name to set in order to override `config.Analytics.CircuitBreakerConfig.MinimumSampleThreshold`.
265+
AnalyticsCircuitBreakerMinimumSampleThresholdEnvVarKey = "SIDE_PROJECT_ANALYTICS_CIRCUIT_BREAKER_MINIMUM_SAMPLE_THRESHOLD"
266+
267+
// AnalyticsPosthogCircuitBreakingErrorRateEnvVarKey is the environment variable name to set in order to override `config.Analytics.Posthog.CircuitBreakerConfig.ErrorRate`.
268+
AnalyticsPosthogCircuitBreakingErrorRateEnvVarKey = "SIDE_PROJECT_ANALYTICS_POSTHOG_CIRCUIT_BREAKING_ERROR_RATE"
269+
270+
271+
/*
272+
...and so on and so forth
273+
*/
274+
)
275+
```
276+
277+
Which documents not just every valid environment variable, but also the field name that it manipulates. As with any generated code in this project, this file [is checked for consistency in CI](https://blog.verygoodsoftwarenotvirus.ru/posts/generated-files/), so I can count on this being accurate and up to date, and I’m free to rename and move fields around at my leisure.
278+
279+
This took me maybe a day or so to do, and if I never encounter a need to make use of this, it will technically have been wasted time. I know myself well enough, however, to know that if I experienced failing to change a value with an outdated environment variable that I had populated by hand, I’d be cursing myself for not spending the day. If I never endeavored to document it, and I had to trawl through this expansive config keeping prefixes in mind in order to deduce what the environment variable was, I’d be cursing myself for not spending the day.
280+
281+
Instead, I took the day to give myself the gift of good tooling, and I can spend subsequent days worrying about the problems I’m actually trying to solve, and not letting toil get in the way of progress.

0 commit comments

Comments
 (0)