Need to investigate how rsync handles symbolic links or if this is a handicap when scraping html.
This can be a major issue when for example the Rocky mirror is synchronized via http and the links 8/ and 9/ are pointing to the highest released version, httpsync would then copy and maintaining x2 of these versions on local disk wish is very unnecessary and takes up lots of disk space.
But the links could also be a construct of the filelist file that is able to list types as d/f/l where "l" is the symbolic link, but the information is received elsewhere.
Look into how this can be done via curl or other solution in httpsync script to avoid duplicate file structures.
Need to investigate how rsync handles symbolic links or if this is a handicap when scraping html.
This can be a major issue when for example the Rocky mirror is synchronized via http and the links 8/ and 9/ are pointing to the highest released version, httpsync would then copy and maintaining x2 of these versions on local disk wish is very unnecessary and takes up lots of disk space.
But the links could also be a construct of the filelist file that is able to list types as d/f/l where "l" is the symbolic link, but the information is received elsewhere.
Look into how this can be done via curl or other solution in httpsync script to avoid duplicate file structures.