SscsEx : (An attempt to) create a Simple SFTP Client/Serveur (SSCS) written in Elixir (Ex)...

---

👉 PRIOR AND VERY IMPORTANT NOTICE !!! 👈

• SscsEx is only intended to give some coding hints to those who are exploring the same matters (SSH/SFTP exchanges) in Elixir and it will probably NEVER be updated.
• SscsEx surely has BUGS: So, USE IT at your OWN RISKS with NO GARANTEE AT ALL !
• I'm an Elixir n00b: who learned this (fabulous) language for only several monthes. So be indulgent !
• English is NOT my native language: Beeing French is a very good excuse as the time spent to learn such a difficult/full of exceptions/full of oddities language as mine is lost to correctly learn others... Especially for an old and lazy bloke as me ! ;)

---

Now that's done, let's go !

SscsEx is a basic sftp server with a client part added to allow files transfers between two SSCS servers. Authentification for connection from a SSCS server to another SSCS server uses public keys only. The transfers are always processed by streaming files (chunk by chunk). To avoid integrity problems, checksums of source and destination files are computed and compared for each transfer. When a transfer fails, resume can happen. If a SSCS server is (normally) stopped, the failed transfers are saved on a disk database (DETS) and when the server restart, this disk database is loaded in memory (ETS) and these transfers are then resumed. If some part of the destination file is already present and its size is less than the source file size, the difference is appended to the destination file, whatever could the OVERWRITE option (see below) be.

The Elixir files of this project can be compiled as an Elixir "release" to make a bundle which can be copied and directly launched on a computer with the same OS, even if Elixir or Erlang are not installed on it. More informations about releases on the official documentation page

Particularities :

• Elixir tasks are initiated by the GenServer.cast functions to avoid the wait for a transfer to complete before another one is launched (asynchronous transfers). Returns are received by the handle_info functions.

• A file, named sscs_tranfers.db (DETS file), containing the failed transfers datas, is created or read when the SSCS server is launched. This file is located in the root folder of the application.

• The HTML documentation, generated by ExDoc, is available in the doc folder. The @specs (giving the type of each argument) are not set, except for the SscsEx.SFTP module.

• No other library than the standard Erlang :ssh and :ssh_sftp libraries has been used as dependencies for the SSH/SFTP part.

Dependencies (when MIX_ENV = prod):

Only:

• Elixir_uuid: To generate unique Ids for transfers,
• Logger File Backend: To enable logging in the console and in a file.

System variables :

The following variables:

• SSCS_PORT (Integer : 0-65536) = listening port of the SSCS server (default = 5555),
• SSCS_LOG_FILE (String) = complete path to the log file (default = subfolder/file "/log/sscs.log" of the release),
• SSCS_ROOT_DIR (String) : root folder to send or receive files (default = subfolder "/sftp" of the release),
• SSCS_AUTH_DIR (String) : folder containing the authorized keys (default = subfolder "/sftp/.ssh" of the release),
• SSCS_SYSTEM_DIR (String) : folder containing the server SSH keys (default = subfolder "/sftp/ssh_daemon" of the release),
• SSCS_STREAM_BYTES (Integer): Size of chunks streamed, in bytes (default = 2 Mo),
• SSCS_CHECKSUM_TYPE (String : "sha", "sha224" "sha256","sha384","sha512","sha3_224", "sha3_256","sha3_384","sha3_512", "blake2b", "blake2s", "md5", "md4"): checksum type used (default = "sha256").
• SSCS_OVERWRITE (String: "true" or "false"): Enable or disable overwriting destination files if they exist. No retry attempted if with SSCS_OVERWRITE set to false (default = "false"),
• SSCS_RETRIES (Integer): max retries to a failed transfer (default = 3),
• SSCS_TEMP_RETRY (Integer): time, in milli-seconds, between two retry attempts (default = 180000, 3 minutes).

can be set before launching the SSCS server.

The SSCS server will log simultaneously in the SSCS_LOG_FILE and in the console.

Examples :

14:39:09.708 [info] SUCCESS: Transfer_id = "8ac1d2c6-bfcc-4412-867f-3d771ead3ecb" : Local file = c:/SSCS/sftp/success.txt sent to : Remote file = tr/success.txt : Remote server = 'mydistantSSCSserver':5555 : Average transfer rate = 0.0 Mb/s : Checksum (type: sha256 ) for both files = C784C2AF93C512DA33C363A2E3D8CEAFE45D3A80B88B762CE015914B00D496AD

or (with SSCS_OVERWRITE set to "false"):

10:51:48.459 [error] FAILURE : Transfer_id = "8e0eb430-42d5-49ce-b472-4ccfd876c476" : Local file = c:/SSCS/sftp/failure.txt WAS NOT sent to : Remote file = tr/failure.txt: Remote server = 'localhost':5000} : "Upload error: Remote file tr/failure.txt already exists." NO RETRY permitted !

or:

10:54:48.528 [error] FAILURE (Retry 1): Transfer_id = "8e0eb430-42d5-4412-867f-3d771ead3ecb" : Local file = c:/SSCS/sftp/resume.txt WAS NOT sent to : Remote file = tr/resume.txt : Remote server = 'localhost':5000} : "Connection error - Reason: :econnrefused" Retrying later...

10:57:48.559 [error] FAILURE (Retry 2): Transfer_id = "8e0eb430-42d5-4412-867f-3d771ead3ecb" : Local file = c:/SSCS/sftp/resume.txt WAS NOT sent to : Remote file = tr/resume.txt : Remote server = 'localhost':5000} : "Connection error - Reason: :econnrefused" Retrying later...

11:00:48.590 [info] : Transfer_id = "8e0eb430-42d5-49ce-b472-4ccfd876c476" : Local file = c:/SSCS/sftp/resume.txt was sent to : Remote file = tr/resume.txt : Remote server = 'localhost':5000} : Average transfer rate = 0.0 Mb/s : Checksum (type: sha256 ) for both files = 3EEE6DBF789756FAF9C9BCE07FEB38FEAF221E83E4FA83D61D39A6B63D47F335

SSCS available functions :

• SscsEx.start("","") : Start the server.
• SscsEx.stop() : Stop the server.
• SscsEx.get_conf() : Display the current configuration (SSH, transferts, logging) of the application, as an array.
• SscsEx.display_failed_transfers() : Display the failed transfers list.
• SscsEx.clean_failed_transfers() : Empty the failed transfers list.
• SscsEx.ping('<host_name>',<port>) : Check if a SSCS server is reachable.
• SscsEx.send_file(local_path, remote_path, host, port): Send/Upload a file.
• SscsEx.get_file(local_path, remote_path, host, port): Get/Download a file.
• SscsEx.send_file_append(local_path, remote_path, host, port) : Append the exceeding part of local file to the remote file, with checksum checks (WARNING !: No verification done a priori to ensure that the remote file is the first part of the local file !!!).
• SscsEx.get_file_append(local_path, remote_path, host, port) : Append the exceeding part of remote file to the local file, with checksum checks (WARNING !: No verification done a priori to ensure that the local file is the first part of the remote file !!!).

with:

• local_path(String surrounded by ' or ", ) = complete path of the local file, relative to the SSCS_ROOT_DIR,
• remote_path (String surrounded by ' or ", ) = complete path of the remote file, relative to the SSCS_ROOT_DIR,
• host (String surrounded by ' or ", ) = name of remote server (IP address or DNS name),
• port (integer: 0 to 65535) = listening port of SSCS remote server.

SSCS server launch (using a release) :

<release folder>\bin\SSCS_ex start (*)

The console should display:

[info] SSCS server started on port 5555...

Transfers launches (console or script)

Open another console then type :

<release folder>\bin\SSCS_ex rpc "<function>(<arguments>)"

example: SSCS_ex rpc "SscsEx.send_file('mysourcefile.ext', 'mydestinationfile.ext', 'myremotehost', 5555)"

(Properly) stop the SSCS server

<release folder>\bin\SSCS_ex stop (*)

(*): To see all the availables options depending on the OS, go to the official documentation page of Mix Release

On Windows, commands are:

• start: Start the SSCS server,
• start_iex: Start the SSCS server in an IEx console (Elixir interpreter),
• install : Install the SSCS server as a Windows service,
• rpc "FUNC": Executes the "FUNC" function,
• restart: Restart the SSCS server,
• stop: Stop the SSCS server,
• pid: Display the SSCS server PID,
• version: Display the SSCS server version.

Credits :

Great thanks to:

• Michael Dorman for his Sftp_Ex library,
• Codenaut for his Exsftpd library,
• Tobias Casper for his SFTP Client library,
• All the people who kindly gave me help on the Elixir Forum.