wt/backup
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat: Implement full network functionality with HTTP and P2P modes
Go Build and Test / build-and-test (pull_request) Successful in 37s
Details

Browse Source 
This major update introduces a robust, multi-faceted networking
capability, allowing backups to be managed through a central server or
via direct peer-to-peer connections.

- **Add `serve` command**: Implemented a new `serve` command that runs a
persistent HTTP server. This allows for a centralized backup management
workflow where multiple clients can upload and download archives. -
**HTTP Client Mode**: The `create` and `restore` commands can now
function as HTTP clients, targeting a `serve` instance using the
`ip:port/filename` address format.

- **Restore P2P Mode**: The original peer-to-peer streaming
functionality (where `restore` acts as a TCP server and `create` as a
client) has been fully restored and integrated into the new, more robust
command architecture. - **Accurate Network Progress**: Fixed a critical
bug where network progress bars would not display correctly. The `serve`
command now sends a `Content-Length` header, and `restore` is refactored
to handle asynchronous state updates correctly. - **Smart Address
Parsing**: The application now automatically distinguishes between HTTP
mode (`ip:port/filename`) and P2P mode (`ip:port`), providing a seamless
user experience.

- **Update `README.md`**: The project's README has been completely
rewritten to document all three operating modes (Local, HTTP Server,
P2P) with clear usage examples. - **Add Gitea Actions Workflow**: A new
CI pipeline (`.gitea/workflows/build-and-test.yml`) has been added to
automatically build and test the application on every push and pull
request.
This commit is contained in:
wt committed 2025-08-03 10:46:14 +07:00
1 parent 668721ab58
commit 6b1a302c18
7 files changed
+543 -240
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+84 -44
View File
@@ -1,57 +1,56 @@
# Go Backup Utility
A simple yet powerful command-line utility written in Go for creating and restoring backups of files, directories, and block devices (like disks). It supports standard `gzip` compression for files/disks and `tar.gz` for directories.
A simple yet powerful command-line utility written in Go for creating, restoring, and serving backups of files, directories, and block devices. It supports `gzip` compression for files/disks and `tar.gz` for directories, with multiple modes for network operations.
The key feature of this tool is the ability to stream backups directly over the network, allowing you to create a backup on one machine and restore it on another without storing intermediate files.
The key feature of this tool is the ability to handle backups over the network, allowing you to create a backup on one machine and restore it on another without storing intermediate files, either via a direct connection or through a central backup server.
## Features
- **File, Disk, and Directory Backup**: Create backups of individual files, entire directories, or full block devices (e.g., `/dev/sda`).
- **Network Streaming**: Directly stream backups between two machines over a TCP connection.
- `create` acts as a network client, sending data.
- `restore` acts as a network server, receiving data.
- **Multiple Network Modes**:
1. **HTTP Server Mode**: Run a dedicated server using the `serve` command to upload and download backup files.
2. **Peer-to-Peer (P2P) Mode**: Directly stream a backup from a machine running `create` to another running `restore`.
- **Compression**: Uses `gzip` for efficient compression. Directories are archived using `tar` before compression.
- **Interactive Progress Bars**: Displays a real-time progress bar for both creating and restoring, so you always know the status of the operation.
- **Interactive Progress Bars**: Displays a real-time progress bar for creating and restoring, so you always know the status of the operation.
- **Cross-Platform**: Written in Go, it can be compiled for Linux, macOS, and Windows.
## Installation
To build the utility from source, you need to have Go installed on your system.
To build the utility from source, you need to have Go and the `git` client installed.
1. Clone the repository (or download the source code).
2. Navigate to the project directory.
3. Build the binary:
1. Clone the repository:
```sh
git clone https://github.com/your-username/backup-utility.git
cd backup-utility
```
2. Build the binary:
```sh
go build
```
4. This will create an executable file named `backup` (or `backup.exe` on Windows) in the current directory. You can move this file to a directory in your system's `PATH` (e.g., `/usr/local/bin`) to make it accessible from anywhere.
3. This will create an executable file named `backup` (or `backup.exe` on Windows). You can move this file to a directory in your system's `PATH` (e.g., `/usr/local/bin`) to make it globally accessible.
## Usage
The utility has two main commands: `create` and `restore`.
The utility has three main commands: `create`, `restore`, and `serve`.
### Local Backup and Restore
### 1. Local Backup and Restore
#### 1. Backup a Directory
This will create a `tar.gz` archive of the specified source directory.
This mode works entirely on your local filesystem.
#### Backup a Directory
Creates a `tar.gz` archive of the source directory.
```sh
./backup create -s /path/to/source/directory -t /path/to/backup.tar.gz
```
#### 2. Restore a Directory
This will extract the contents of a `tar.gz` archive into the specified target directory. If the target directory is omitted, it defaults to the current working directory.
#### Restore a Directory
Extracts a `tar.gz` archive. If the target directory is omitted, it defaults to the current working directory.
```sh
./backup restore -s /path/to/backup.tar.gz -t /path/to/restore/location
```
#### 3. Backup a File or Disk
This will create a compressed `gz` file from a source file or a block device.
#### Backup a File or Disk
Creates a compressed `.gz` file from a source file or block device.
```sh
# Backup a file
./backup create -s /path/to/largefile.log -t /path/to/largefile.log.gz
@@ -60,10 +59,8 @@ This will create a compressed `gz` file from a source file or a block device.
./backup create -s /dev/sdb1 -t /path/to/sdb1_backup.img.gz
```
#### 4. Restore a File or Disk
This will decompress a `.gz` file to a target file or write the uncompressed data to a block device.
#### Restore a File or Disk
Decompresses a `.gz` file or writes the uncompressed data to a block device.
```sh
# Restore a file
./backup restore -s /path/to/largefile.log.gz -t /path/to/restored_largefile.log
@@ -72,40 +69,83 @@ This will decompress a `.gz` file to a target file or write the uncompressed dat
./backup restore -s /path/to/sdb1_backup.img.gz -t /dev/sdb1
```
### Network Backup and Restore
---
This is the most powerful feature. It allows you to pipe a backup from a source machine (client) to a target machine (server).
### 2. HTTP Server Mode
**Step 1: On the receiving machine (Server)**
This mode is ideal for managing backups in a centralized location. One machine runs `serve`, and other machines can upload (`create`) or download (`restore`) backups from it.
First, run the `restore` command. It will start listening on the specified IP address and port, waiting for a connection. Use `0.0.0.0` to listen on all available network interfaces.
**Step 1: Start the Backup Server**
On your server machine, run the `serve` command. This will start an HTTP server that listens for backup requests.
```sh
# The server waits for data on port 8080 and will restore it to /mnt/data
# Run the server, listening on port 8080
# Backups will be stored in the default directory (~/backups)
./backup serve -a 0.0.0.0:8080
# Specify a custom directory for backups
./backup serve -a 0.0.0.0:8080 -d /mnt/backups
```
The server is now running and waiting for client connections.
**Step 2 (Client): Create and Upload a Backup**
On a client machine, use the `create` command and point the target (`-t`) to the server's address, followed by a `/` and the desired filename for the backup.
```sh
# Back up a directory and upload it to the server as "my_project_backup.tar.gz"
./backup create -s ./my_project -t 192.168.1.100:8080/my_project_backup.tar.gz
# Back up a disk and upload it
./backup create -s /dev/sdc -t 192.168.1.100:8080/sdc_image.img.gz
```
**Step 3 (Client): Download and Restore a Backup**
To restore, use the `restore` command and point the source (`-s`) to the server URL.
```sh
# Download and restore a directory backup from the server
./backup restore -s 192.168.1.100:8080/my_project_backup.tar.gz -t ./restored_project
# Download and restore a disk image
./backup restore -s 192.168.1.100:8080/sdc_image.img.gz -t /dev/sdd
```
---
### 3. Peer-to-Peer (P2P) Mode
This mode streams a backup directly from one machine to another without a central server. It's useful for one-off migrations or direct transfers.
**Step 1: On the receiving machine (Server-like role)**
Run the `restore` command with a listening address (`ip:port`) as the source (`-s`). Use `0.0.0.0` to listen on all network interfaces.
```sh
# The machine waits for data on port 8080 and will restore it to /mnt/data
./backup restore -s 0.0.0.0:8080 -t /mnt/data
```
> The command will print `Waiting for connection on 0.0.0.0:8080...` and will pause until the client connects.
The command will print `Listening on 0.0.0.0:8080...` and will pause until the client connects.
**Step 2: On the sending machine (Client)**
Now, run the `create` command, pointing the target (`-t`) to the IP address and port of the server machine.
**Step 2: On the sending machine (Client-like role)**
Run the `create` command, pointing the target (`-t`) to the IP address and port of the receiving machine.
```sh
# This will back up the /home/user/documents directory and send it directly
# to the server at 192.168.1.100 on port 8080.
./backup create -s /home/user/documents -t 192.168.1.100:8080
# to the machine at 192.168.1.200 on port 8080.
./backup create -s /home/user/documents -t 192.168.1.200:8080
```
The backup process will start, and you will see a progress bar on both the client and server terminals.
The backup will start immediately, and you will see a progress bar on both terminals.
## Command-line Flags
- `create`:
- `-s, --source`: The source file, directory, or disk to back up (required).
- `-t, --target`: The target backup file or network address (`ip:port`) (required).
- `-t, --target`: The target. Can be a local file path, a P2P address (`ip:port`), or an HTTP server URL (`ip:port/filename`) (required).
- `restore`:
- `-s, --source`: The source archive file or network address (`ip:port`) to restore from (required).
- `-t, --target`: The target file, disk, or directory. Required for network sources and file/disk restores. Defaults to the current directory for local archive restores.
- `-s, --source`: The source. Can be a local archive, a P2P listening address (`ip:port`), or an HTTP server URL (`ip:port/filename`) (required).
- `-t, --target`: The target file, disk, or directory. Required for network restores and local file/disk restores.
- `serve`:
- `-a, --address`: The address and port for the server to listen on (defaults to `localhost:8080`).
- `-d, --directory`: The directory to store backups in (defaults to `~/backups`).
## License