Dependencies management

[Arvolear]

Describe the feature

Motivation

Since the simplicity compiler will soon add functionality to support modules and use directive, Simplex needs to manage smart contract dependencies for user convenience and code reuse. SimplicityHL will also be shipped with an std that many projects would want to import easily.

Spec

Installation

A user will specify required dependencies in the simplex.toml file or by calling the simplex install <repo + tag> command.

The example config:

[dependencies]
std = { repo = "BlockstreamResearch/simf-std@v0.0.1", path = "lib/simf-std/contracts" } 

If the dependency is "external".

or

[dependencies]
math = { path = "node_modules/simf-math/contracts" }

If the dependency is "local" or has been downloaded via cargo/npm/etc.

The simplex install command should append the new dependency to the simplex.toml file.

The user may only specify the tag of the dependency to be downloaded. The dependency gets git cloned, but if git submodules are found effective, we may switch to this approach later.

Building

To abstract away the remappings management from the user, Simplex has to figure out the required paths on its own.

The simplex build command has to be expanded to generate artifacts with the remappings constant in place so that the inner program could use them for pruning.

The resolver struct needs to be expanded to resolve all simplex.toml aliases, including the ones present in the simplex.toml of each dependency. Then these aliases can be passed to the generator together with the source paths.

It is important to filter out any files WITHOUT a main function to not generate artifacts for them. In the future, the artifacts generator will need to find all "main" files from all the dependencies. For now, only a shallow scan will suffice.

[stringhandler]

I am worried that we are doing too much with Simplex.toml. It contains regtest and wallet seeds and appears to be used for testing. Dependencies however will be used for any code used by Simplicity, and should IMO not require users to use simplex. We already have users running NodeJS and compiling Simplicity using SimplicityHL as WASM, and also Go transpilers.

My vote would be for a separate file, that describes the SimplicityHL package (similar to Cargo.toml or package.json). Something like Simf.toml or Package.toml or any of the names described here.

Just a clarification: I am not against simplex downloading the libraries in this new package descriptor. I am advocating for a processor that can download the packages and output to raw simplicity. Currently Simplex only compiles programs to rust. My vote would be to use simc for all dependency management and "linking" (it is not linking in the true sense). This will enable users to only have to install a single binary or for non-rust users, allow them to compile simplicity from a single dll or wasm.