Skip to content

feat(configio): implement Builder#61

Merged
GregoryAlbouy merged 10 commits into
mainfrom
feat/configio-builder
Mar 28, 2023
Merged

feat(configio): implement Builder#61
GregoryAlbouy merged 10 commits into
mainfrom
feat/configio-builder

Conversation

@GregoryAlbouy

@GregoryAlbouy GregoryAlbouy commented Nov 1, 2022

Copy link
Copy Markdown
Member

Description

Meant to replace configio.Representation for external use. configio.Representation is hardly usable because of nested structs and pointers: it should remain an internal unmarshaling/marshaling structure only.

Problem statement

A common use-case we have is the need to combine benchttp configuration, by overriding successively different sources (e.g. in benchttp/cli: DefaultRunner <- configuration file <- cli flags).
To do this, we need to know which fields are set and which are not for each override. However, due to Go initializing unset fields with zero values, it's impossible to determine whether a zero-value field was actually set or defaulted by Go.

Consecutive solutions

➡️ Solution 1: runner.Config.Override

To overcome this we first added memorizing capabilities to runner.Config, which used with its method Override allowed to perform this kind of selective override.
This solution was far from ideal: it was cumbersome to use for the consumer (having to keep track of set fields), it polluted package runner with many ConfigFieldXxx declarations and added undesirable complexity to the package.

➡️ Solution 2: configio.Representation

(previously configparse.Representation)

In #58, we nuke the runner.Config structure and its overriding logics.
Instead we take advantage of configio.Representation: previously used as a decoding structure only, it uses pointer fields to distinguish zero values from undefined values. This combined with the adoptions of Marshal-like signature provides a functional overriding logic:

Overriding a benchttp.Runner with json config using configio.Representation
var jsonConfig = []byte(`"runner":{"requests": 42}`)

func main() {
	// start with default runner as base
	runner := benchttp.DefaultRunner()

	// parse json config as configio.Representation
	repr := configio.Representation{}
	_ = configio.Unmarshal(jsonConfig, &repr)

	// override default with json config
	_ = repr.Into(&runner)
}

This works well for what it was initially intended for: decoding bytes. Other than that it turns out to be extremely painful to manipulate, due to nested structs and pointer fields. As such, it ties overriding logics to the action of decoding. Let's try to assign a value to configio.Representation.Body (e.g. in a testing context or from any benchttp consumer point of view):

Cumbersome manipulation of configio.Representation
package main

import "github.com/benchttp/sdk/configio"

func main() {
	repr := configio.Representation{}

	// Nested structs must be redefined
	repr.Request.Body = &struct {
		// Yes, including struct tags
		Type    string `yaml:"type" json:"type"`
		Content string `yaml:"content" json:"content"`
	}{
		Type:    "raw",
		Content: "hello",
	}

	// Pointer values must be stored in a variable first
	//   repr.Request.Method = "GET"
	//                         ^^^^^
	//   cannot use "GET" (untyped string constant) as *string value
	requestMethod := "GET"
	repr.Request.Method = &requestMethod
	runnerConcurrency := 10
	repr.Runner.Concurrency = &runnerConcurrency
}

This shows how unadapted configio.Representation is for the general purpose of building a config. It is tied to the purpose of decoding a JSON/YAML config and as such it must remain an internal decoding structure.

➡️ Solution 3: configio.Builder

The purpose of configio.Builder is to provide a building API for benchttp.Runner.

  • Simple and familiar: it works similarly to other builders (e.g. strings.Builders) using write and setter methods (b.WriteJSON, b.SetConcurrency)
  • It memorizes all changes made via these methods by itself: it does not require to keep track of set fields nor does it use configio.Representation for that matter

This makes configio.Representation obsolete as an exported value, but still needed as an internal decoding structure.

That's why we also unexport configio.Representation and update all related public function to work with configio.Builder instead.

Changes

  • Implement configio.Builder
  • Update decoding functions so they work with configio.Builder instead of configio.Representation
  • Unexport configio.Representation

Notes

@GregoryAlbouy GregoryAlbouy added enhancement New feature or request breaking changes Introduces breaking changes labels Nov 1, 2022
@GregoryAlbouy GregoryAlbouy added this to the v0.2 milestone Nov 1, 2022
@codecov-commenter

codecov-commenter commented Nov 2, 2022

Copy link
Copy Markdown

Codecov Report

Merging #61 (861609c) into main (a729645) will increase coverage by 0.47%.
The diff coverage is 83.33%.

❗ Current head 861609c differs from pull request most recent head 39fb4a6. Consider uploading reports for the commit 39fb4a6 to get more accurate results

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main      #61      +/-   ##
==========================================
+ Coverage   81.16%   81.64%   +0.47%     
==========================================
  Files          28       29       +1     
  Lines         945     1024      +79     
==========================================
+ Hits          767      836      +69     
- Misses        147      150       +3     
- Partials       31       38       +7     
Flag Coverage Δ
unittests 81.64% <83.33%> (+0.47%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
configio/decoder.go 63.63% <50.00%> (-14.15%) ⬇️
configio/yaml.go 91.48% <75.00%> (+5.48%) ⬆️
configio/builder.go 82.71% <82.71%> (ø)
configio/representation.go 68.54% <85.71%> (+0.51%) ⬆️
configio/file.go 92.59% <100.00%> (ø)
configio/json.go 94.44% <100.00%> (+7.26%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@GregoryAlbouy GregoryAlbouy mentioned this pull request Nov 6, 2022
8 tasks
@GregoryAlbouy GregoryAlbouy force-pushed the feat/configio-builder branch 2 times, most recently from 840c662 to 5e9268f Compare November 6, 2022 16:47
@GregoryAlbouy GregoryAlbouy marked this pull request as ready for review November 6, 2022 21:33
Base automatically changed from feat/configio to main March 28, 2023 21:30
Previous benchmarks showed 2.5x better performance over piped modifier.
- Builder.Into -> Builder.Mutate
- Builder.modifiers -> Builder.mutations
- configio.Representation -> configio.representation
- make all representation methods private
- remove Decode methods from interface Decoder and its implementations
- rename DecodeRunner methods to Decode
- remove UnmarshalXXXX functions
- rename UnmarshalXXXXRunner functions to UnmarshalXXXX
@GregoryAlbouy GregoryAlbouy force-pushed the feat/configio-builder branch from e865353 to 39fb4a6 Compare March 28, 2023 21:34
@GregoryAlbouy GregoryAlbouy merged commit 13e624c into main Mar 28, 2023
@GregoryAlbouy GregoryAlbouy deleted the feat/configio-builder branch March 28, 2023 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking changes Introduces breaking changes enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants