Features to implement for 0.2.0 and other tasks

[joelself]

For 0.2.0 I'm looking at adding:

• Key/value delete
• Key/value append insert
• Key modification
• Table and Array of Table add insert
• Table and Array of Table delete
• Table and Array of Table modify
• More error reporting
• Remove dependence on regexes to allow tomllib to build on stable
• First tomlkit release

Since I'm moving soon and the prep and actual move will be taking so much time, I'm going to limit 0.2.0 features to just what's needed to get cargo-edit up and running. However I'm adding a change to let tomllib to compile on stable. There's only 4 regexes I need to eliminate for that to happen.

2016-06-14: Adding tomlkit release since there's interest in it and it shouldn't be too bad (there's not a lot of functionality to implement).

[vi]

No point for tomlkit?

[joelself]

I aiming for this to be a really quick release with some key features, but it looks like that's not going to happen because I'm in the middle of moving.

[joelself]

I guess since this won't be as quick of a release as I had planned I could add in tomlkit if people (or even just one person) want it. Do you mind if I ask what you plan to use it for (you probably already know, but the library doesn't have a whole lot of functionality yet.

Edit: Thinking about it more, since there's so little to implement, maybe tomlkit should get a version 0.1.2 release.

Edit: I guess it would be 0.1.2 since there's already a 0.1.1.

[vi]

I though about killercup/cargo-edit#86 (comment) (doesn't mean I will actually implement it).

Any way, I don't think a git config (which may happen to work on Cargo.toml because of similarity to INI format), but for toml is a useless tool. It may find use outside Rust.

[vi]

Another use: I was curious which Cargo packages reside inside Rust itself:

rust$ for i in $(find . -name 'Cargo.toml'); do git config -f "$i" -l 2> /dev/null | grep package.name; done
package.name=rustdoc
package.name=fmt_macros
package.name=rustc_save_analysis
package.name=rustc_back
package.name=rustc_lint
package.name=rustc_llvm
package.name=std_shim
package.name=test_shim
package.name=libc
package.name=rustc-main
package.name=error_index_generator
package.name=cargotest
package.name=rustbook
package.name=linkchecker
package.name=getopts
package.name=syntax
package.name=core
package.name=generate-files
package.name=libc-test
package.name=libc
package.name=rand
package.name=rbml
package.name=term
package.name=test
package.name=build_helper
package.name=rustc_const_eval
package.name=rustc_const_math
package.name=rustc_driver
package.name=rustc_platform_intrinsics
package.name=serialize
package.name=syntax_ext
package.name=rustc_passes
package.name=rustc_plugin
package.name=rustc_unicode
package.name=alloc_jemalloc
package.name=log
package.name=std
package.name=rustc_typeck
package.name=rustc_borrowck
package.name=collections
package.name=rustc_front
package.name=rustc_metadata
package.name=rustc_trans
package.name=rustc_bitflags
package.name=alloc
package.name=arena
package.name=flate
package.name=graphviz
package.name=rustc_mir
package.name=rustc
package.name=rustc_data_structures
package.name=rustc_privacy
package.name=alloc_system
package.name=rustc_resolve
package.name=bootstrap

I expect tomlkit (or something like that) be better tool for a job rather than misused git config.

[vi]

Also such tool (especially for getting and enumerating values) may be useful if somebody wants to create toml config for a Bash script.

Typically such confics are also shell scripts (sourced into the program), and as configs may be edited by non-programmers, it may lead to a trouble. For example, changing DO_SOMETING=no into DO_SOMETHING= yes in a sourced config makes /usr/bin/yes program to be executed instead of setting the value of DO_SOMETHING because of extra space character.

[joelself]

./target/debug/tomlkit --set-value package.keywords[2],'FOO',basic-string --print-doc --file Cargo.toml
Success

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>DOCUMENT<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

[package]
name = "tomllib"
version = "0.1.2"
authors = ["Joel Self <self@jself.com>"]
description = "A format-preserving TOML file parser and manipulator"
license = "MIT"
repository = "https://github.com/joelself/tomllib"
readme = "README.md"
documentation = "https://jself.io/rustdocs/tomllib/0.1.1/tomllib/"
keywords = ["toml", "parser", "FOO", "decode", "nom"]
exclude = ["assets/*", "tests/*"]

[dependencies.nom]
version = "^1.2.0"
features = ["regexp"]

[dependencies.regex]
version = "^0.1.48"

# Uncomment these, and comment out the regexp feature to turn on statically compiled regexes
#features = ["regexp_macros"]
#[dependencies.regex_macros]
[dependencies.log]
version = "^0.3.5"
[dependencies.env_logger]
version = "^0.3.2"
[dependencies.pirate]
version = "1.0.0"
[dependencies.csv]
version = "0.14.4"

[[bin]]
name = "tomlkit"

It's basically done. What's left is:

• Set/get/check multiple values in one command
• Option to choose output on one line in CSV format or one value per line.
• Quiet mode option for set-value, right now it prints Success. End-users might not want that.

[vi]

I tried installing it with cargo install tomllib and got panicked at 'not yet implemented'.

Will there be a command to list content of entire toml file in line-oriented format (with or without values), like in git config -f ... -l?

[joelself]

It's not completely finished or released yet. When it's released you'll see a version number bump to 0.1.2 or actually it might be a completely different crate. I'm not quite sure how that works right now. If you really want to try it you can check out the 0.1.2 branch and compile it and run it. It has get-value, has-value, get-children, has-children, set-value commands right now, but they each only take a single key to lookup/change.
I hadn't planned on adding a command to list all keys and values, but it shouldn't be hard to add.

[joelself]

Almost done. Only thing left is to actually write to the file if you're setting a value.

• Write to file on successful set-value

[joelself]

It's ready for its first release. Here's how you would get the package name, package version, whether or not it has a license and all of the package's dependencies in one short command:

find . -name 'Cargo.toml' | ./toml_parser/target/debug/tomlkit -g package.name,package.version --has-value package.license -c dependencies
"methodtest", "0.1.0", false, [nom]
"mtailtest", "0.1.0", false, [mtaillib]
"multitail", "0.1.0", false, [term, notify, getopts]
"mtaillib", "0.1.0", false, [notify, libc]
"nomplusplus", "0.1.0", true, [radix_trie, regex, lazy_static]
"rproxy", "0.1.0", false, [radix_trie]
"radix_trie", "0.0.8", true, [nibble_vec]
"rustfmt", "0.2.1", true, [toml, rustc-serialize, unicode-segmentation, regex, term, strings, diff, syntex_syntax, log, env_logger, getopts]
"test", "0.1.0", false, []
"tomllib", "0.1.2", true, [nom, regex, log, env_logger, pirate, csv]
"tomllib", "0.1.0", true, [nom, regex, log, env_logger]
"tomllib", "0.1.1", true, [nom, regex, log, env_logger]
"trietest", "0.1.0", false, [radix_trie]

[vi]

That does not look UNIX-y. Output format looks CSV-ish. It requires further parsing to be usable in shell scripting.

As a minimum there should be a command to dump string value without quotes.

Maybe tomlkit should not try to handle multiple queries in one run, instead providing a good interface to get/set just one value.

That is how I see the fragment above:

for i in $(find . -name 'Cargo.toml'); do 
    echo "P"
    tomlkit -f "$i" -g package.name
    tomlkit -f "$i" -g package.version
    tomlkit -f "$i" --has-value package.license && echo has license || echo no licence
    tomlkit -f "$i" -c dependencies
done

P
methodtest
0.1.0
no licence
nom
P
mtailtest
0.1.0
no lisence
mtaillib
P
multitail
0.1.0
no licence
term
notify
getopts
P
mtaillib
0.1.0
no license
notify
libc
...