add docker copy_config_files profile field

[davidsanfal]

Changelog: Feature: Add copy_config_files setting to the [runner] profile section (Docker runner), allowing to copy extra host config files/folders into the container, opt-in and disabled by default.
Docs: https://github.com/conan-io/docs/pull/XXXX
Closes: #20281

Adds a new opt-in copy_config_files key to the [runner] profile section, accepting a comma-separated list of fnmatch patterns relative to the Conan home folder, e.g.:

[runner]
type=docker
copy_config_files=extensions/*

It's opt-in and empty by default, so existing profiles/behavior are unaffected.

[memsharded]

The "package storage" folder (p) seems to be missing here.
Also, the version.txt that declares the version installed in the machine should probably never be copied either.

[davidsanfal]

the package storage folder (p by default, but relocatable via core.cache:storage_path) is excluded where it is used.

storage_path = Path(self.conan_api._api_helpers.cache.store)
if storage_path.is_relative_to(self.conan_api.home_folder):
    excludes.append(f"{storage_path.relative_to(self.conan_api.home_folder).as_posix()}/*")

[memsharded]

That is my question, why not just ignoring the default p subfolder?, instead of such automatic exclusion computation including a check to see if it is relative or not?

[davidsanfal]

That was changed in the latest commit, and I also added a rule to always exclude the download_cache too

[czoido]

For the same reason shouldn't we also skip credentials.json and source_credentials.json and maybe adding those explicitly to the copy would override this skip?

[czoido]

I think this is not testing the feature is working, this will also run for the host, before the docker even runs?

[davidsanfal]

I thought about letting explicit patterns override the default excludes, but that falls apart for wildcard excludes like .local_recipes_index/* (a request for .local_recipes_index/foo just wouldn't match it), and getting the general case right would mean fiddly glob intersection logic that's easy to get wrong. So instead I made the exclude list itself a new copy_config_files_excludes setting, defaulting to what we have now. If someone needs to bring back a file, they just set it explicitly in their profile, no hidden override rules to reason about.

[czoido]

I'm not sure about adding another configuration like copy_config_files_excludes What about two groups instead of copy_config_files_excludes?

• Always excluded, not overridable: version.txt and .local_recipes_index/*.
• Sensitive, excluded unless listed by exact name in copy_config_files: .conan.db, credentials.json, source_credentials.json.

All wildcard excludes are in the fixed group, so no glob intersection is needed, and we keep a single setting. With the current approach, copy_config_files_excludes=.conan.db,source_credentials.json (to copy credentials.json) replaces the whole default list, so version.txt and .local_recipes_index/ get copied too unless the user remembers to list them again.

[czoido]

this checks tend to be fragile, wouldn't it be easier to check:

assert "conan-runner-docker | my custom profile plugin running" in client.out

[franramirez688]

Do we really need this copy_config_files_excludes? I think this makes the code more complex. In my mind, the use case would be:

copy_config_files=extensions/*

copy_config_files=extensions/*,otherfolder/* 

Or even

copy_config_files=*   # copy all the non-excluded files

If the user wants to copy an excluded file deliberately, then they should put it in the settings explicitly:

copy_config_files=extensions/*,credentials.json

[czoido]

I suggested doing something like this: #20332 (comment)

[franramirez688]

Oh, I did not see your comment above 🙏