Skip to content

docs: refresh README with up-to-date usage #75

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
woutervanranst wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs: refresh README with up-to-date usage #75

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
256 changes: 138 additions & 118 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,159 +11,179 @@
[![Arius.Core Version](https://img.shields.io/nuget/v/WouterVanRanst.Arius.Core?logo=nuget)](https://www.nuget.org/packages/WouterVanRanst.Arius.Core)
[![ClickOnce](https://img.shields.io/badge/Windows-ClickOnce-dsfs?logo=windows&logoColor=lightblue)](https://woutervanranst.github.io/Arius/Arius.Explorer.application)

Arius is a lightweight archival solution, specifically built to leverage the Azure Blob Archive tier.

The name derives from the Greek for 'immortal'.

## Why?

1. **Supporting 3-2-1 Backup Strategy**: Arius offers a secure and cost-effective offsite backup solution, complementing offline disk backups. The use of Azure Blob Archive tier allows for low-cost archiving, approximately 1 EUR per TB per month.

1. **Single Pane of Glass**: Arius ensures seamless visibility to offline backups without requiring a separate application. By creating small "pointers" locally, they remain present in the local filesystem and easily discoverable in Windows Explorer.

1. **Encryption**: For those with privacy concerns, Arius uses AES256/openssl-compatible encryption, providing robust client-side encryption.

1. **Deduplication**: Arius does not store duplicate files or file parts. It offers file-level deduplication by default and allows optional variable block size deduplication within files.
Arius is a cross-platform archival solution that keeps cold backups affordable without giving up visibility into your files. It encrypts and deduplicates data locally, stores the content in Azure Blob Storage (Archive/Cool/Cold/Hot tiers), and leaves *pointer* files behind so your backup still looks like a regular folder structure.

---

- [Key features](#key-features)
- [Quick start](#quick-start)
- [Prerequisites](#prerequisites)
- [Get the CLI](#get-the-cli)
- [Configure Azure Storage](#configure-azure-storage)
- [Archive data](#archive-data)
- [Restore data](#restore-data)
- [Command reference](#command-reference)
- [Archive](#archive)
- [Restore](#restore)
- [Running in Docker](#running-in-docker)
- [Arius Explorer (Windows UI)](#arius-explorer-windows-ui)
- [Documentation](#documentation)
- [Manual disaster recovery](#manual-disaster-recovery)
- [Architecture](#architecture)

## Key features

* **3-2-1 friendly off-site backups** – Archive cold data into low-cost Azure Blob tiers while keeping your production storage clean.
* **Client-side encryption** – AES-256 encryption happens before anything leaves your machine; only encrypted blobs reach Azure.
* **Chunk-level deduplication** – Identical content is stored only once, even when filenames or locations differ.
* **Pointer-aware file system integration** – Tiny `.arius.pointer` files keep your directory tree browsable and searchable.
* **Automatable CLI & Docker support** – Run Arius interactively, on a schedule, or in containers as part of your backup pipeline.

![](docs/overview.png)

## How
## Quick start

Arius is a command-line tool (CLI) that can be run manually or scheduled. It can also be run as a Docker container.
### Prerequisites

Arius Explorer is a Windows application that offers a graphical user interface into an Arius repository.
1. [.NET SDK 9.0 or later](https://dotnet.microsoft.com/download) (for running or building the CLI locally).
2. An Azure Storage account with [hierarchical namespace disabled](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-namespace) and the Archive tier available.
3. A secure passphrase that will be used to encrypt and decrypt your blobs.

### Arius Explorer
### Get the CLI

![](docs/arius.explorer.png)
* **Download a build:** Pre-built binaries for Linux, macOS, and Windows are published on the [GitHub Releases](https://github.com/woutervanranst/Arius/releases) page.
* **Build from source:**
```bash
git clone https://github.com/woutervanranst/Arius.git
cd Arius
dotnet publish src/Arius.Cli -c Release -r linux-x64 --self-contained false -o ./publish
./publish/arius --version
```
Adjust `-r` for your OS (`win-x64`, `osx-arm64`, ...). Replace `--self-contained false` with `true` if you want a single-file build that includes the runtime.

Install it via ClickOnce by clicking on the shield at the top of the page.
### Configure Azure Storage

The status icon is a combination of the state in the local file system (left half moon) and the state in the repository (right half moon). For example:
1. Create (or reuse) a blob container that will store Arius data.
2. Generate a storage account access key (Key 1 or Key 2 in the Azure Portal).
3. Optionally set environment variables so you don't have to pass credentials on every run:
```bash
export ARIUS_ACCOUNT_NAME=<storage-account-name>
export ARIUS_ACCOUNT_KEY=<storage-account-key>
```

| Icon | Local file systen | Azure Repository |
| --- |---|---|
| ![](docs/status/NNYC.png) | Not present | Pointer and Binary |
| ![](docs/status/NYYC.png) | Only the Pointer | Pointer and Binary |
| ![](docs/status/YYYC.png) | Pointer and Binary | Pointer and Binary |
### Archive data

```bash
arius archive /path/to/data \
--accountname <storage-account-name> \
--accountkey <storage-account-key> \
--container <container-name> \
--passphrase <encryption-passphrase>
```

### Arius CLI
During the run Arius will:

#### Archive to Azure
* hash and deduplicate data
* compress and encrypt before upload
* write progress updates to the console
* leave `.arius.pointer` files in place of archived files (unless `--remove-local` is specified)

CLI:
Logs are written to `%LOCALAPPDATA%/Arius/logs` on Windows and `~/.local/share/Arius/logs` on Linux/macOS. When running in Docker the logs are written to `/logs`.

<!-- TODO How can they install the CLI? -->
### Restore data

```
arius archive <path>
--accountname <accountname>
--accountkey <accountkey>
--passphrase <passphrase>
--container <containername>
[--remove-local]
[--tier=<hot/cool/archive>]
[--dedup]
[--fasthash]
```bash
arius restore --root /restore/location "Photos/2020" "Documents/report.docx" \
--accountname <storage-account-name> \
--accountkey <storage-account-key> \
--container <container-name> \
--passphrase <encryption-passphrase> \
--download --include-pointers
```

CLI in Docker:

```
docker run
-v <absolute_path_to_archive>:/archive
[-v <absolute_path_to_logs>:/logs]
woutervanranst/arius

archive
--accountname <accountname>
--accountkey <accountkey>
--passphrase <passphrase>
--container <containername>
[--remove-local]
[--tier=<hot/cool/archive>]
[--dedup]
[--fasthash]
* `--download` hydrates blobs from Azure and writes the binary content locally.
* `--include-pointers` recreates pointer files next to restored binaries so the folder stays "Arius-aware".
* Omitting `--download` performs a *sync-only* run that updates/creates pointer files without pulling full data.

## Command reference

### Archive

| Parameter | Description | Notes |
| --- | --- | --- |
| `<local-root>` | Directory to archive. | A pointer file replaces each archived file unless `--remove-local` is used. |
| `--accountname`, `-n` | Azure Storage account name. | Can also be supplied via `ARIUS_ACCOUNT_NAME`. |
| `--accountkey`, `-k` | Azure Storage account key. | Can also be supplied via `ARIUS_ACCOUNT_KEY`. |
| `--container`, `-c` | Blob container that will hold chunks, pointers, and state. | Required. |
| `--passphrase`, `-p` | Encryption passphrase. | Required – keep it safe. |
| `--tier` | Storage tier for uploaded chunks. | Defaults to `archive`. Supported values: `hot`, `cool`, `cold`, `archive`. |
| `--remove-local` | Delete local binaries after successful upload. | Pointer files remain so the folder stays browsable. |

### Restore

| Parameter | Description | Notes |
| --- | --- | --- |
| `--root`, `-r` | Local root folder to use for restores. | Defaults to the current working directory. Required when running from outside the target folder. |
| `Targets` | Optional paths (files or directories) under the root to restore. | Defaults to `./` (entire repository). |
| `--accountname`, `-n` | Azure Storage account name. | Supports `ARIUS_ACCOUNT_NAME`. |
| `--accountkey`, `-k` | Azure Storage account key. | Supports `ARIUS_ACCOUNT_KEY`. |
| `--container`, `-c` | Azure Blob container to read from. | Required. |
| `--passphrase`, `-p` | Passphrase used during archive. | Required. |
| `--download` | Retrieve the binary content. | Without this flag the command only syncs pointer files. |
| `--include-pointers` | Keep or recreate pointer files alongside restored binaries. | Recommended for ongoing Arius use. |

### Running in Docker

The official container image (`woutervanranst/arius`) contains the CLI and expects bind mounts for both your data and optional logs.

```bash
docker run --rm \
-v /absolute/path/to/data:/archive \
-v /absolute/path/to/logs:/logs \
woutervanranst/arius archive \
--accountname <storage-account-name> \
--accountkey <storage-account-key> \
--container <container-name> \
--passphrase <passphrase>
```

### Restore from Azure
For restore runs, replace `archive` with `restore` and add `--download` / `--include-pointers` as needed. The container always works inside `/archive`, so you do not pass a positional path parameter.

CLI:
## Arius Explorer (Windows UI)

```
arius restore <path>
--accountname <accountname>
--accountkey <accountkey>
--passphrase <passphrase>
--container <containername>
[--synchronize]
[--download]
[--keep-pointers]
```

CLI in Docker:
![](docs/arius.explorer.png)

```
docker run
-v <absolute_path_to_archive>:/archive
[-v <absolute_path_to_logs>:/logs]
woutervanranst/arius

restore
--accountname <accountname>
--accountkey <accountkey>
--passphrase <passphrase>
--container <containername>
[--synchronize]
[--download]
[--keep-pointers]
```
Arius Explorer provides a Windows desktop experience on top of the same repository:

### Arguments
1. Install it via the [ClickOnce installer](https://woutervanranst.github.io/Arius/Arius.Explorer.application).
2. Sign in with the same storage account, container, and passphrase that you use for the CLI.
3. Browse pointer status – icons combine local state (left half) and cloud state (right half).

| Argument | Description | Notes |
| - | - | - |
| path | The path on the local file system | For `archive`:<br>The root directory to archive<br><br>For `restore`:<ul><li>If path is a directory: restore all pointer files in the (sub)directories.<li>If path is a file: restore this file.</ul>
| logpath | Path to the folder to store the logs | OPTIONAL. NOTE: Only for Docker.
| &#x2011;&#x2011;accountname, &#x2011;n | Storage Account Name
| &#x2011;&#x2011;accountkey, &#x2011;k | [Storage Account Key](https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage?tabs=azure-portal) | Can be set through:<ul><li>Argument<li>Environment variable `ARIUS_ACCOUNT_KEY`<li>Docker environment variable `ARIUS_ACCOUNT_KEY`</ul>
| &#x2011;&#x2011;passphrase, &#x2011;p | Passphrase with which the blobs are encrypted
| &#x2011;&#x2011;container, &#x2011;c | Blob container to use | OPTIONAL. Default: 'arius'.
| &#x2011;&#x2011;remove-local | Remove local file after a successful upload | `archive`-only<br> OPTIONAL. Default: Local files are not deleted after archiving.
| &#x2011;&#x2011;tier | [Blob storage tier (hot/cool/archive)](https://docs.microsoft.com/en-us/azure/storage/blobs/access-tiers-overview) | `archive`-only<br> OPTIONAL. Default: 'archive'.
| &#x2011;&#x2011;dedup | Deduplicate on block level | `archive`-only<br> OPTIONAL. Default: deduplicate on file level.
| &#x2011;&#x2011;fasthash | When a pointer file is present, use that hash instead of re-hashing the full file again | `archive`-only<br> OPTIONAL. Default: false.<br>NOTE: Do **NOT** use this if the contents of the files are modified. Arius will not pick up the changes.
| &#x2011;&#x2011;synchronize | Bring the structure of the local file system (pointer files) in line with the latest state of the remote repository | `restore`-only<br> OPTIONAL. Default: do not synchronize.<br>This command only touches the pointers (ie. `.pointer.arius` files). Other files are left untouched:<ul><li>Pointers that exist in the archive but not locally are created.<li>Pointers that exist locally but not in the archive are deleted</ul>
| &#x2011;&#x2011;download | Download and restore the actual file (contents) | `restore`-only<br> OPTIONAL. Default: do not download.<br>NOTE: If the file is in the archive blob tier, hydration to an online tier is started. Run the restore command again after ~15 hours to download the file.
| &#x2011;&#x2011;keep-pointers | Keep pointer files after downloading content files | `restore`-only<br>OPTIONAL. Default: keep the pointers.
| Icon | Local file system | Azure repository |
| --- | --- | --- |
| ![](docs/status/NNYC.png) | Not present | Pointer and binary |
| ![](docs/status/NYYC.png) | Pointer only | Pointer and binary |
| ![](docs/status/YYYC.png) | Pointer and binary | Pointer and binary |

### Restoring manually
## Documentation

For detailed instructions on restoring files manually, _without_ using any of the Arius tools, please consult the [Restore Manually](docs/manualrestore.md) section.
Additional technical documentation lives in the [`docs/`](docs) folder:

## Advanced
* [Archive command deep dive](docs/ArchiveCommand.md)
* [Storage abstractions](docs/Storage.md)
* [Versioning model](docs/versioning.md)
* [State repository internals](docs/StateRepository.md)

### Deduplication
## Manual disaster recovery

Arius employs deduplication to optimize storage efficiency. For detailed explanations, diagrams, and benchmarks on deduplication, refer to the [Deduplication](docs/deduplication.md) section in the documentation.
Need to recover data without Arius? Follow the step-by-step [manual restore guide](docs/manualrestore.md) to decrypt blobs and recreate files directly from Azure Storage Explorer.

## Technical Documentation
## Architecture

### Dependencies

![](http://www.plantuml.com/plantuml/proxy?cache=no&src=https://raw.githubusercontent.com/woutervanranst/Arius/main/docs/dependencies.puml)

### Arius.Core Architecture
### Arius.Core architecture

![](docs/AriusFlows-Arius.Core%20Structure.drawio.svg)

### Arius.Core Domain Model

![](http://www.plantuml.com/plantuml/proxy?cache=no&src=https://raw.githubusercontent.com/woutervanranst/Arius/main/docs/classdiagram.puml)

# Attributions

Arius Icon by [Freepik](https://www.flaticon.com/free-icon/iceberg_2055379?related_id=2055379).

Many thanks to [ReSharper](https://www.jetbrains.com/community/opensource/#support) and [PostSharp](https://www.postsharp.net/metalama/pricing) for their open source generosity.