Skip to content

Conversation

@diamondburned
Copy link

This commit changes both JSONToYAML and YAMLToJSON to preserve the map
key ordering when the document is transformed. This is done by
reimplementing the library to use YAML v3's yaml.Node API.

This vastly improves the Go test output. The benchmark is also added for
the next commit which changes the behavior of this library and may
potentially be marginally slower.
@diamondburned
Copy link
Author

Benchmark results:

goos: linux
goarch: amd64
pkg: github.com/invopop/yaml
cpu: AMD Ryzen 5 5500                               
              │ /tmp/old.bench │           /tmp/new.bench            │
              │     sec/op     │   sec/op     vs base                │
JSONToYAML-12      6.656µ ± 4%   5.811µ ± 1%  -12.69% (p=0.000 n=10)
YAMLToJSON-12      4.688µ ± 3%   4.927µ ± 1%   +5.10% (p=0.001 n=10)
geomean            5.585µ        5.351µ        -4.21%

(old = before PR; new = after PR)

This commit changes both JSONToYAML and YAMLToJSON to preserve the map
key ordering when the document is transformed. This is done by
reimplementing the library to use YAML v3's yaml.Node API.
@diamondburned
Copy link
Author

Some of this work was borrowed from @silasdavis and @skipor from PR ghodss/yaml#62.

diamondburned added a commit to diamondburned/kin-openapi that referenced this pull request Aug 12, 2024
This commit adds the `OrderedPropertyKeys` method to the
`openapi3.Schema`:

    OrderedPropertyKeys returns the keys of the properties in the order
    they were defined. This is useful for generating code that needs to
    iterate over the properties in a consistent order. If the keys could
    not be extracted for some reason, then this method automatically
    sorts the keys to be deterministic.

This is done via a temporary fork of the YAML-to-JSON transformation library.
It will not be ready until invopop/yaml#13 is merged.
diamondburned added a commit to diamondburned/kin-openapi that referenced this pull request Aug 12, 2024
This commit adds the `PropertyKeys` property to the type
`openapi3.Schema` which contains the keys of the `Properties` map in the
order that they appear in the original YAML file. This is useful to
guarantee deterministic code generation.

This is done via a temporary fork of the YAML-to-JSON transformation library.
It will not be ready until invopop/yaml#13 is merged.
diamondburned added a commit to diamondburned/kin-openapi that referenced this pull request Aug 13, 2024
This commit adds the `PropertyKeys` property to the type
`openapi3.Schema` which contains the keys of the `Properties` map in the
order that they appear in the original YAML file. This is useful to
guarantee deterministic code generation.

This is done via a temporary fork of the YAML-to-JSON transformation library.
It will not be ready until invopop/yaml#13 is merged.
@lzap
Copy link

lzap commented Apr 25, 2025

What is the status @samlown I understand you guys patched the security problem and there is that. Do you plan to push new features? I am interested in this and the date parsing problem reported the other day as well. Cheers!

@samlown
Copy link
Contributor

samlown commented Apr 25, 2025

@lzap I honestly didn't have this PR on my radar. We actively use this package in other projects, so our plan is to maintain as much as possible, we're just super-low on time. It looks like @diamondburned did a really good job here, so I'll get it on my Todos.

@samlown
Copy link
Contributor

samlown commented Apr 25, 2025

So this one get a bit more complicated. The go-yaml project on which this is based has been archived 😞 Seems like the best option would be to migrate away to a different solution rather than update to something that is no longer active.

@lzap
Copy link

lzap commented Apr 25, 2025

I do not think there is need to migrate off this library, k8s folks already forked it and are fully committed to do security maintenance (no new features): https://github.com/kubernetes-sigs/yaml/tree/master/goyaml.v3 that should be pretty straightforward replacement.

@samlown
Copy link
Contributor

samlown commented Apr 25, 2025

Nice. I was actually thinking of doing something similar and stripping out the bits we don't need. However, it looks like the underlying project there is a direct fork of the same ghodss/yaml package this one is forked from! I suspect the Kubernetes team are better at maintaining this than me, it could be a much better option.

@lzap
Copy link

lzap commented Apr 30, 2025

However, it looks like the underlying project there is a direct fork of the same ghodss/yaml package this one is forked from!

Oh I missed that :-D

Well whatever happens, I am very interested in preserving map ordering, it is a massive pain. @diamondburned would you mind opening this PR at k8s to see how responsive they are? Chances are they might be opinionated on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants