Merge pull request 'feat: Implement full network functionality with HTTP and P2P modes' (#1) from develop into main
Go Build and Test / build-and-test (push) Successful in 23s

Reviewed-on: https://gitea.home.com:3000/TolaMironcenko/backup/pulls/1
This commit was merged in pull request #1.
This commit is contained in:
wt committed 2025-08-03 10:46:54 +07:00
commit f2e670f66a
7 files changed
+543 -240

No files matched your search

+41
View File
@@ -0,0 +1,41 @@
name: Go Build and Test
# This workflow runs on pushes and pull requests to the main branches.
on:
push:
branches:
- main
- master
pull_request:
branches:
- main
- master
jobs:
build-and-test:
# You might need to change 'ubuntu-latest' to a tag that matches your self-hosted Gitea runner, e.g., 'linux', 'go', etc.
runs-on: ubuntu-latest
steps:
# Step 1: Check out the repository code
- name: Checkout
uses: actions/checkout@v4
# Step 2: Set up the Go environment
- name: Set up Go
uses: actions/setup-go@v5
with:
# Use a Go version that matches your project's go.mod file
go-version: '1.22'
# Step 3: Download Go module dependencies
- name: Tidy and Vendor
run: go mod tidy && go mod vendor
# Step 4: Build the application to ensure it compiles
- name: Build
run: go build -v ./...
# Step 5: Run tests to check for regressions
# This step will pass if no tests are found, which is fine for now.
- name: Test
run: go test -v ./...
+84 -44
View File
@@ -1,57 +1,56 @@
# Go Backup Utility # Go Backup Utility
A simple yet powerful command-line utility written in Go for creating and restoring backups of files, directories, and block devices (like disks). It supports standard `gzip` compression for files/disks and `tar.gz` for directories. A simple yet powerful command-line utility written in Go for creating, restoring, and serving backups of files, directories, and block devices. It supports `gzip` compression for files/disks and `tar.gz` for directories, with multiple modes for network operations.
The key feature of this tool is the ability to stream backups directly over the network, allowing you to create a backup on one machine and restore it on another without storing intermediate files. The key feature of this tool is the ability to handle backups over the network, allowing you to create a backup on one machine and restore it on another without storing intermediate files, either via a direct connection or through a central backup server.
## Features ## Features
- **File, Disk, and Directory Backup**: Create backups of individual files, entire directories, or full block devices (e.g., `/dev/sda`). - **File, Disk, and Directory Backup**: Create backups of individual files, entire directories, or full block devices (e.g., `/dev/sda`).
- **Network Streaming**: Directly stream backups between two machines over a TCP connection. - **Multiple Network Modes**:
- `create` acts as a network client, sending data. 1. **HTTP Server Mode**: Run a dedicated server using the `serve` command to upload and download backup files.
- `restore` acts as a network server, receiving data. 2. **Peer-to-Peer (P2P) Mode**: Directly stream a backup from a machine running `create` to another running `restore`.
- **Compression**: Uses `gzip` for efficient compression. Directories are archived using `tar` before compression. - **Compression**: Uses `gzip` for efficient compression. Directories are archived using `tar` before compression.
- **Interactive Progress Bars**: Displays a real-time progress bar for both creating and restoring, so you always know the status of the operation. - **Interactive Progress Bars**: Displays a real-time progress bar for creating and restoring, so you always know the status of the operation.
- **Cross-Platform**: Written in Go, it can be compiled for Linux, macOS, and Windows. - **Cross-Platform**: Written in Go, it can be compiled for Linux, macOS, and Windows.
## Installation ## Installation
To build the utility from source, you need to have Go installed on your system. To build the utility from source, you need to have Go and the `git` client installed.
1. Clone the repository (or download the source code). 1. Clone the repository:
2. Navigate to the project directory. ```sh
3. Build the binary: git clone https://github.com/your-username/backup-utility.git
cd backup-utility
```
2. Build the binary:
```sh ```sh
go build go build
``` ```
4. This will create an executable file named `backup` (or `backup.exe` on Windows) in the current directory. You can move this file to a directory in your system's `PATH` (e.g., `/usr/local/bin`) to make it accessible from anywhere. 3. This will create an executable file named `backup` (or `backup.exe` on Windows). You can move this file to a directory in your system's `PATH` (e.g., `/usr/local/bin`) to make it globally accessible.
## Usage ## Usage
The utility has two main commands: `create` and `restore`. The utility has three main commands: `create`, `restore`, and `serve`.
### Local Backup and Restore ### 1. Local Backup and Restore
#### 1. Backup a Directory This mode works entirely on your local filesystem.
This will create a `tar.gz` archive of the specified source directory.
#### Backup a Directory
Creates a `tar.gz` archive of the source directory.
```sh ```sh
./backup create -s /path/to/source/directory -t /path/to/backup.tar.gz ./backup create -s /path/to/source/directory -t /path/to/backup.tar.gz
``` ```
#### 2. Restore a Directory #### Restore a Directory
Extracts a `tar.gz` archive. If the target directory is omitted, it defaults to the current working directory.
This will extract the contents of a `tar.gz` archive into the specified target directory. If the target directory is omitted, it defaults to the current working directory.
```sh ```sh
./backup restore -s /path/to/backup.tar.gz -t /path/to/restore/location ./backup restore -s /path/to/backup.tar.gz -t /path/to/restore/location
``` ```
#### 3. Backup a File or Disk #### Backup a File or Disk
Creates a compressed `.gz` file from a source file or block device.
This will create a compressed `gz` file from a source file or a block device.
```sh ```sh
# Backup a file # Backup a file
./backup create -s /path/to/largefile.log -t /path/to/largefile.log.gz ./backup create -s /path/to/largefile.log -t /path/to/largefile.log.gz
@@ -60,10 +59,8 @@ This will create a compressed `gz` file from a source file or a block device.
./backup create -s /dev/sdb1 -t /path/to/sdb1_backup.img.gz ./backup create -s /dev/sdb1 -t /path/to/sdb1_backup.img.gz
``` ```
#### 4. Restore a File or Disk #### Restore a File or Disk
Decompresses a `.gz` file or writes the uncompressed data to a block device.
This will decompress a `.gz` file to a target file or write the uncompressed data to a block device.
```sh ```sh
# Restore a file # Restore a file
./backup restore -s /path/to/largefile.log.gz -t /path/to/restored_largefile.log ./backup restore -s /path/to/largefile.log.gz -t /path/to/restored_largefile.log
@@ -72,40 +69,83 @@ This will decompress a `.gz` file to a target file or write the uncompressed dat
./backup restore -s /path/to/sdb1_backup.img.gz -t /dev/sdb1 ./backup restore -s /path/to/sdb1_backup.img.gz -t /dev/sdb1
``` ```
### Network Backup and Restore ---
This is the most powerful feature. It allows you to pipe a backup from a source machine (client) to a target machine (server). ### 2. HTTP Server Mode
**Step 1: On the receiving machine (Server)** This mode is ideal for managing backups in a centralized location. One machine runs `serve`, and other machines can upload (`create`) or download (`restore`) backups from it.
First, run the `restore` command. It will start listening on the specified IP address and port, waiting for a connection. Use `0.0.0.0` to listen on all available network interfaces. **Step 1: Start the Backup Server**
On your server machine, run the `serve` command. This will start an HTTP server that listens for backup requests.
```sh ```sh
# The server waits for data on port 8080 and will restore it to /mnt/data # Run the server, listening on port 8080
# Backups will be stored in the default directory (~/backups)
./backup serve -a 0.0.0.0:8080
# Specify a custom directory for backups
./backup serve -a 0.0.0.0:8080 -d /mnt/backups
```
The server is now running and waiting for client connections.
**Step 2 (Client): Create and Upload a Backup**
On a client machine, use the `create` command and point the target (`-t`) to the server's address, followed by a `/` and the desired filename for the backup.
```sh
# Back up a directory and upload it to the server as "my_project_backup.tar.gz"
./backup create -s ./my_project -t 192.168.1.100:8080/my_project_backup.tar.gz
# Back up a disk and upload it
./backup create -s /dev/sdc -t 192.168.1.100:8080/sdc_image.img.gz
```
**Step 3 (Client): Download and Restore a Backup**
To restore, use the `restore` command and point the source (`-s`) to the server URL.
```sh
# Download and restore a directory backup from the server
./backup restore -s 192.168.1.100:8080/my_project_backup.tar.gz -t ./restored_project
# Download and restore a disk image
./backup restore -s 192.168.1.100:8080/sdc_image.img.gz -t /dev/sdd
```
---
### 3. Peer-to-Peer (P2P) Mode
This mode streams a backup directly from one machine to another without a central server. It's useful for one-off migrations or direct transfers.
**Step 1: On the receiving machine (Server-like role)**
Run the `restore` command with a listening address (`ip:port`) as the source (`-s`). Use `0.0.0.0` to listen on all network interfaces.
```sh
# The machine waits for data on port 8080 and will restore it to /mnt/data
./backup restore -s 0.0.0.0:8080 -t /mnt/data ./backup restore -s 0.0.0.0:8080 -t /mnt/data
``` ```
> The command will print `Waiting for connection on 0.0.0.0:8080...` and will pause until the client connects. The command will print `Listening on 0.0.0.0:8080...` and will pause until the client connects.
**Step 2: On the sending machine (Client)** **Step 2: On the sending machine (Client-like role)**
Run the `create` command, pointing the target (`-t`) to the IP address and port of the receiving machine.
Now, run the `create` command, pointing the target (`-t`) to the IP address and port of the server machine.
```sh ```sh
# This will back up the /home/user/documents directory and send it directly # This will back up the /home/user/documents directory and send it directly
# to the server at 192.168.1.100 on port 8080. # to the machine at 192.168.1.200 on port 8080.
./backup create -s /home/user/documents -t 192.168.1.100:8080 ./backup create -s /home/user/documents -t 192.168.1.200:8080
``` ```
The backup will start immediately, and you will see a progress bar on both terminals.
The backup process will start, and you will see a progress bar on both the client and server terminals.
## Command-line Flags ## Command-line Flags
- `create`: - `create`:
- `-s, --source`: The source file, directory, or disk to back up (required). - `-s, --source`: The source file, directory, or disk to back up (required).
- `-t, --target`: The target backup file or network address (`ip:port`) (required). - `-t, --target`: The target. Can be a local file path, a P2P address (`ip:port`), or an HTTP server URL (`ip:port/filename`) (required).
- `restore`: - `restore`:
- `-s, --source`: The source archive file or network address (`ip:port`) to restore from (required). - `-s, --source`: The source. Can be a local archive, a P2P listening address (`ip:port`), or an HTTP server URL (`ip:port/filename`) (required).
- `-t, --target`: The target file, disk, or directory. Required for network sources and file/disk restores. Defaults to the current directory for local archive restores. - `-t, --target`: The target file, disk, or directory. Required for network restores and local file/disk restores.
- `serve`:
- `-a, --address`: The address and port for the server to listen on (defaults to `localhost:8080`).
- `-d, --directory`: The directory to store backups in (defaults to `~/backups`).
## License ## License
+93 -84
View File
@@ -7,9 +7,11 @@ import (
"fmt" "fmt"
"io" "io"
"net" "net"
"net/http"
"os" "os"
"path/filepath" "path/filepath"
"runtime" "runtime"
"strings"
"syscall" "syscall"
"time" "time"
"unsafe" "unsafe"
@@ -46,31 +48,30 @@ var createCmd = &cobra.Command{
} }
type model struct { type model struct {
progress progress.Model progress progress.Model
source string source string
target string target string
totalBytes int64 totalBytes int64
processed int64 processed int64
done bool done bool
err error err error
startTime time.Time startTime time.Time
progressCh chan int64 progressCh chan int64
isDirectory bool isDirectory bool
isNetworkTarget bool httpTargetInfo *ServerTargetInfo
isP2PTarget bool // Флаг для старого P2P-режима
} }
type progressMsg int64 type progressMsg int64
type doneMsg struct{} type doneMsg struct{}
type errorMsg struct{ err error } type errorMsg struct{ err error }
// getPathSize возвращает размер файла, диска или директории
func getPathSize(path string) (int64, error) { func getPathSize(path string) (int64, error) {
fileInfo, err := os.Stat(path) fileInfo, err := os.Stat(path)
if err != nil { if err != nil {
return 0, err return 0, err
} }
// Для Linux: проверка, является ли источник блочным устройством
if runtime.GOOS == "linux" { if runtime.GOOS == "linux" {
stat, ok := fileInfo.Sys().(*syscall.Stat_t) stat, ok := fileInfo.Sys().(*syscall.Stat_t)
if ok && (stat.Mode&syscall.S_IFMT) == syscall.S_IFBLK { if ok && (stat.Mode&syscall.S_IFMT) == syscall.S_IFBLK {
@@ -94,7 +95,6 @@ func getPathSize(path string) (int64, error) {
} }
} }
// Для директорий
if fileInfo.IsDir() { if fileInfo.IsDir() {
var total int64 var total int64
err := filepath.Walk(path, func(_ string, info os.FileInfo, err error) error { err := filepath.Walk(path, func(_ string, info os.FileInfo, err error) error {
@@ -109,7 +109,6 @@ func getPathSize(path string) (int64, error) {
return total, err return total, err
} }
// Для обычных файлов
return fileInfo.Size(), nil return fileInfo.Size(), nil
} }
@@ -129,15 +128,30 @@ func initialModel(src, dst string) (*model, error) {
return nil, err return nil, err
} }
isDirectory := srcInfo.IsDir() isDirectory := srcInfo.IsDir()
networkTarget := isNetworkAddress(dst)
targetInfo, isHttp := parseServerTarget(dst)
isP2P := false
targetPath := dst targetPath := dst
if !networkTarget { if isHttp {
// Новый HTTP-режим
if isDirectory && !strings.HasSuffix(targetInfo.Filename, ".tar.gz") {
targetInfo.Filename += ".tar.gz"
} else if !isDirectory && !strings.HasSuffix(targetInfo.Filename, ".gz") {
targetInfo.Filename += ".gz"
}
targetInfo.URL = fmt.Sprintf("http://%s/backup/%s", targetInfo.Address, targetInfo.Filename)
targetPath = targetInfo.URL
} else if isNetworkAddress(dst) {
// Старый P2P-режим
isP2P = true
} else {
// Локальный файл
if isDirectory { if isDirectory {
if filepath.Ext(targetPath) != ".tar.gz" { if filepath.Ext(targetPath) != ".tar.gz" {
targetPath += ".tar.gz" targetPath += ".tar.gz"
} }
} else { // Это файл или диск } else {
if filepath.Ext(targetPath) != ".gz" { if filepath.Ext(targetPath) != ".gz" {
targetPath += ".gz" targetPath += ".gz"
} }
@@ -145,14 +159,15 @@ func initialModel(src, dst string) (*model, error) {
} }
return &model{ return &model{
progress: p, progress: p,
source: src, source: src,
target: targetPath, target: targetPath,
totalBytes: totalBytes, totalBytes: totalBytes,
startTime: time.Now(), startTime: time.Now(),
progressCh: make(chan int64, 100), progressCh: make(chan int64, 100),
isDirectory: isDirectory, isDirectory: isDirectory,
isNetworkTarget: networkTarget, httpTargetInfo: targetInfo,
isP2PTarget: isP2P,
}, nil }, nil
} }
@@ -163,21 +178,47 @@ func (m *model) Init() tea.Cmd {
) )
} }
// getTargetWriter создает io.WriteCloser для файла или сетевого подключения
func (m *model) getTargetWriter() (io.WriteCloser, error) { func (m *model) getTargetWriter() (io.WriteCloser, error) {
if m.isNetworkTarget { if m.httpTargetInfo != nil {
// Новый HTTP-режим
pipeReader, pipeWriter := io.Pipe()
req, err := http.NewRequest("POST", m.httpTargetInfo.URL, pipeReader)
if err != nil {
return nil, fmt.Errorf("failed to create http request: %w", err)
}
req.Header.Set("Content-Type", "application/octet-stream")
req.ContentLength = -1 // Stream upload
go func() {
client := &http.Client{} // No timeout for uploads
resp, err := client.Do(req)
if err != nil {
pipeWriter.CloseWithError(fmt.Errorf("http request failed: %w", err))
return
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
bodyBytes, _ := io.ReadAll(resp.Body)
err := fmt.Errorf("server returned non-200 status: %s\n%s", resp.Status, string(bodyBytes))
pipeWriter.CloseWithError(err)
}
}()
return pipeWriter, nil
} else if m.isP2PTarget {
// Старый P2P-режим
conn, err := net.DialTimeout("tcp", m.target, 10*time.Second) conn, err := net.DialTimeout("tcp", m.target, 10*time.Second)
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to connect to %s: %w", m.target, err) return nil, fmt.Errorf("failed to connect to %s: %w", m.target, err)
} }
// Протокол: сначала отправляем 8 байт (размер) и 1 байт (тип) // Отправляем бинарный заголовок (размер + тип)
// 1. Общий размер (int64)
if err := binary.Write(conn, binary.BigEndian, m.totalBytes); err != nil { if err := binary.Write(conn, binary.BigEndian, m.totalBytes); err != nil {
conn.Close() conn.Close()
return nil, fmt.Errorf("failed to send backup size: %w", err) return nil, fmt.Errorf("failed to send backup size: %w", err)
} }
// 2. Тип бэкапа (1 = директория, 0 = файл/диск)
var typeByte byte = 0 var typeByte byte = 0
if m.isDirectory { if m.isDirectory {
typeByte = 1 typeByte = 1
@@ -186,10 +227,10 @@ func (m *model) getTargetWriter() (io.WriteCloser, error) {
conn.Close() conn.Close()
return nil, fmt.Errorf("failed to send backup type: %w", err) return nil, fmt.Errorf("failed to send backup type: %w", err)
} }
return conn, nil return conn, nil
} }
// Логика для локального файла
// Локальный файл
return os.Create(m.target) return os.Create(m.target)
} }
@@ -216,7 +257,7 @@ func (m *model) startBackup() tea.Msg {
return return
} }
m.progressCh <- -2 // Сигнал завершения m.progressCh <- -2
}() }()
return nil return nil
@@ -229,30 +270,17 @@ func (m *model) backupFileOrDisk(w io.Writer) error {
} }
defer srcFile.Close() defer srcFile.Close()
gzipWriter := gzip.NewWriter(w) progressWriter := &progressTracker{Writer: w, progressCh: m.progressCh}
gzipWriter := gzip.NewWriter(progressWriter)
defer gzipWriter.Close() defer gzipWriter.Close()
buf := make([]byte, 32*1024) // 32KB buffer _, err = io.Copy(gzipWriter, srcFile)
for { return err
n, err := srcFile.Read(buf)
if n > 0 {
if _, writeErr := gzipWriter.Write(buf[:n]); writeErr != nil {
return writeErr
}
m.progressCh <- int64(n)
}
if err == io.EOF {
break
}
if err != nil {
return err
}
}
return nil
} }
func (m *model) backupDirectory(w io.Writer) error { func (m *model) backupDirectory(w io.Writer) error {
gzipWriter := gzip.NewWriter(w) progressWriter := &progressTracker{Writer: w, progressCh: m.progressCh}
gzipWriter := gzip.NewWriter(progressWriter)
defer gzipWriter.Close() defer gzipWriter.Close()
tarWriter := tar.NewWriter(gzipWriter) tarWriter := tar.NewWriter(gzipWriter)
@@ -288,39 +316,22 @@ func (m *model) backupDirectory(w io.Writer) error {
} }
defer srcFile.Close() defer srcFile.Close()
buf := make([]byte, 32*1024) _, err = io.Copy(tarWriter, srcFile)
for { return err
n, err := srcFile.Read(buf)
if n > 0 {
if _, err := tarWriter.Write(buf[:n]); err != nil {
return err
}
m.progressCh <- int64(n)
}
if err == io.EOF {
break
}
if err != nil {
return err
}
}
return nil
}) })
} }
func (m *model) progressListener() tea.Msg { func (m *model) progressListener() tea.Msg {
select { n := <-m.progressCh
case n := <-m.progressCh: if n < 0 {
switch { if n == -1 {
case n == -1: // Ошибка
return errorMsg{m.err} return errorMsg{m.err}
case n == -2: // Завершение
return doneMsg{}
default: // Прогресс
m.processed += n
return progressMsg(m.processed)
} }
return doneMsg{}
} }
m.processed += n
return progressMsg(m.processed)
} }
func (m *model) Update(msg tea.Msg) (tea.Model, tea.Cmd) { func (m *model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
@@ -352,7 +363,6 @@ func (m *model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
m.err = msg.err m.err = msg.err
return m, tea.Quit return m, tea.Quit
} }
return m, nil return m, nil
} }
@@ -363,12 +373,11 @@ func (m *model) View() string {
if m.done { if m.done {
duration := time.Since(m.startTime) duration := time.Since(m.startTime)
// ratio := float64(m.processed) / float64(m.totalBytes) * 100
backupType := "File/disk" backupType := "File/disk"
if m.isDirectory { if m.isDirectory {
backupType = "Directory" backupType = "Directory"
} }
if m.isNetworkTarget { if m.httpTargetInfo != nil || m.isP2PTarget {
backupType += " network" backupType += " network"
} }
@@ -386,7 +395,7 @@ func (m *model) View() string {
if m.isDirectory { if m.isDirectory {
operation = "Archiving" operation = "Archiving"
} }
if m.isNetworkTarget { if m.httpTargetInfo != nil || m.isP2PTarget {
operation = "Sending" operation = "Sending"
} }
@@ -408,7 +417,7 @@ func (m *model) View() string {
func init() { func init() {
rootCmd.AddCommand(createCmd) rootCmd.AddCommand(createCmd)
createCmd.Flags().StringVarP(&source, "source", "s", "", "Source file, directory or disk (e.g., /dev/sda) (required)") createCmd.Flags().StringVarP(&source, "source", "s", "", "Source file, directory or disk (e.g., /dev/sda) (required)")
createCmd.Flags().StringVarP(&target, "target", "t", "", "Target backup file or network address (e.g., /path/to/backup.tar.gz or 127.0.0.1:8080) (required)") createCmd.Flags().StringVarP(&target, "target", "t", "", "Target backup file or network address (e.g., /path/to/backup.tar.gz, 127.0.0.1:8080/backup.gz, or 127.0.0.1:8080) (required)")
createCmd.MarkFlagRequired("source") createCmd.MarkFlagRequired("source")
createCmd.MarkFlagRequired("target") createCmd.MarkFlagRequired("target")
} }
+147 -111
View File
@@ -7,6 +7,7 @@ import (
"fmt" "fmt"
"io" "io"
"net" "net"
"net/http"
"os" "os"
"path/filepath" "path/filepath"
"runtime" "runtime"
@@ -28,7 +29,7 @@ var (
var restoreCmd = &cobra.Command{ var restoreCmd = &cobra.Command{
Use: "restore", Use: "restore",
Short: "Restore a backup", Short: "Restore a backup",
Long: "Restore a backup from a local archive or network stream to a file, disk, or directory.", Long: "Restore a backup from a local archive, P2P stream, or HTTP server.",
Run: func(cmd *cobra.Command, args []string) { Run: func(cmd *cobra.Command, args []string) {
model, err := initialRestoreModel(restoreSource, restoreTarget) model, err := initialRestoreModel(restoreSource, restoreTarget)
if err != nil { if err != nil {
@@ -45,18 +46,27 @@ var restoreCmd = &cobra.Command{
} }
type restoreModel struct { type restoreModel struct {
progress progress.Model progress progress.Model
source string source string
target string target string
totalBytes int64 totalBytes int64
processed int64 processed int64
done bool done bool
err error err error
startTime time.Time startTime time.Time
progressCh chan int64 progressCh chan int64
isDirectory bool isDirectory bool
isNetworkSource bool httpSourceInfo *ServerTargetInfo
statusMessage string isP2PSource bool
statusMessage string
}
// restoreSourceReadyMsg is sent when the source is ready to be read.
// It carries the reader, the total size, and whether it's a directory backup.
type restoreSourceReadyMsg struct {
reader io.ReadCloser
totalBytes int64
isDir bool
} }
type restoreProgressMsg int64 type restoreProgressMsg int64
@@ -64,21 +74,24 @@ type restoreDoneMsg struct{}
type restoreErrorMsg struct{ err error } type restoreErrorMsg struct{ err error }
func initialRestoreModel(src, dst string) (*restoreModel, error) { func initialRestoreModel(src, dst string) (*restoreModel, error) {
isNetwork := isNetworkAddress(src)
p := progress.New( p := progress.New(
progress.WithDefaultGradient(), progress.WithDefaultGradient(),
progress.WithWidth(40), progress.WithWidth(40),
) )
var totalBytes int64 = 1 // Placeholder for network source, to avoid division by zero. var totalBytes int64 = 1 // Placeholder, will be updated.
var isDir bool var isDir, isP2P bool
targetPath := dst targetPath := dst
status := "Initializing..." status := "Initializing..."
if isNetwork { httpInfo, isHttp := parseServerTarget(src)
if targetPath == "" {
return nil, fmt.Errorf("target flag -t is required for a network source") if isHttp {
} isDir = strings.HasSuffix(httpInfo.Filename, ".tar.gz")
status = fmt.Sprintf("Connecting to %s...", httpInfo.Address)
src = httpInfo.URL
} else if isNetworkAddress(src) {
isP2P = true
status = fmt.Sprintf("Listening on %s...", src) status = fmt.Sprintf("Listening on %s...", src)
} else { } else {
fileInfo, err := os.Stat(src) fileInfo, err := os.Stat(src)
@@ -87,89 +100,96 @@ func initialRestoreModel(src, dst string) (*restoreModel, error) {
} }
totalBytes = fileInfo.Size() totalBytes = fileInfo.Size()
isDir = strings.HasSuffix(src, ".tar.gz") isDir = strings.HasSuffix(src, ".tar.gz")
}
if isDir && targetPath == "" { if isDir && targetPath == "" {
targetPath, err = os.Getwd() // For directory restores, we can default to the current directory.
if err != nil { // We do this check here, but also again after getting the P2P type.
return nil, fmt.Errorf("failed to get current directory: %w", err) var err error
} targetPath, err = os.Getwd()
} else if !isDir && targetPath == "" { if err != nil {
return nil, fmt.Errorf("target flag -t is required for file or disk restoration") return nil, fmt.Errorf("failed to get current directory: %w", err)
} }
} else if !isDir && targetPath == "" && !isP2P {
// Target is mandatory for file/disk restores (except P2P, where we can't know the type yet).
return nil, fmt.Errorf("target flag -t is required for this restoration")
} }
return &restoreModel{ return &restoreModel{
progress: p, progress: p,
source: src, source: src,
target: targetPath, target: targetPath,
totalBytes: totalBytes, totalBytes: totalBytes,
startTime: time.Now(), startTime: time.Now(),
progressCh: make(chan int64, 100), progressCh: make(chan int64, 100),
isDirectory: isDir, isDirectory: isDir,
isNetworkSource: isNetwork, httpSourceInfo: httpInfo,
statusMessage: status, isP2PSource: isP2P,
statusMessage: status,
}, nil }, nil
} }
func (m *restoreModel) Init() tea.Cmd { func (m *restoreModel) Init() tea.Cmd {
return tea.Batch( return m.fetchSourceCmd
m.startRestore,
m.restoreProgressListener,
)
} }
func (m *restoreModel) getSourceReader() (io.ReadCloser, error) { func (m *restoreModel) fetchSourceCmd() tea.Msg {
if m.isNetworkSource { if m.httpSourceInfo != nil {
// New HTTP Mode
client := &http.Client{}
resp, err := client.Get(m.source)
if err != nil {
return restoreErrorMsg{fmt.Errorf("failed to connect to server: %w", err)}
}
if resp.StatusCode != http.StatusOK {
bodyBytes, _ := io.ReadAll(resp.Body)
resp.Body.Close()
return restoreErrorMsg{fmt.Errorf("server returned non-200 status: %s\n%s", resp.Status, string(bodyBytes))}
}
return restoreSourceReadyMsg{reader: resp.Body, totalBytes: resp.ContentLength, isDir: m.isDirectory}
} else if m.isP2PSource {
// Old P2P Mode
listener, err := net.Listen("tcp", m.source) listener, err := net.Listen("tcp", m.source)
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to listen on %s: %w", m.source, err) return restoreErrorMsg{fmt.Errorf("failed to listen on %s: %w", m.source, err)}
} }
defer listener.Close() // Close listener after accepting one connection // Accept is blocking, so it must be in a command.
m.statusMessage = fmt.Sprintf("Waiting for connection on %s", m.source)
// This is a blocking call
conn, err := listener.Accept() conn, err := listener.Accept()
listener.Close() // Close listener after one connection.
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to accept connection: %w", err) return restoreErrorMsg{fmt.Errorf("failed to accept connection: %w", err)}
} }
m.statusMessage = "Client connected. Receiving data..." var size int64
if err := binary.Read(conn, binary.BigEndian, &size); err != nil {
// Protocol: Read 8 bytes for size, 1 byte for type
if err := binary.Read(conn, binary.BigEndian, &m.totalBytes); err != nil {
conn.Close() conn.Close()
return nil, fmt.Errorf("failed to read backup size: %w", err) return restoreErrorMsg{fmt.Errorf("failed to read backup size: %w", err)}
} }
if m.totalBytes == 0 {
m.totalBytes = 1 // Avoid division by zero for empty files
}
typeByte := make([]byte, 1) typeByte := make([]byte, 1)
if _, err := io.ReadFull(conn, typeByte); err != nil { if _, err := io.ReadFull(conn, typeByte); err != nil {
conn.Close() conn.Close()
return nil, fmt.Errorf("failed to read backup type: %w", err) return restoreErrorMsg{fmt.Errorf("failed to read backup type: %w", err)}
} }
m.isDirectory = (typeByte[0] == 1) return restoreSourceReadyMsg{reader: conn, totalBytes: size, isDir: typeByte[0] == 1}
return conn, nil
} }
// Logic for local file // Local File Mode
m.statusMessage = fmt.Sprintf("Opening archive %s", m.source) fileInfo, err := os.Stat(m.source)
return os.Open(m.source) if err != nil {
return restoreErrorMsg{err}
}
file, err := os.Open(m.source)
if err != nil {
return restoreErrorMsg{err}
}
return restoreSourceReadyMsg{reader: file, totalBytes: fileInfo.Size(), isDir: m.isDirectory}
} }
func (m *restoreModel) startRestore() tea.Msg { func (m *restoreModel) processRestoreCmd(r io.ReadCloser) tea.Cmd {
go func() { return func() tea.Msg {
sourceReader, err := m.getSourceReader() defer r.Close()
if err != nil { progressReader := &progressTracker{
m.err = err Reader: r,
m.progressCh <- -1
return
}
defer sourceReader.Close()
progressReader := &restoreProgressReader{
reader: sourceReader,
progressCh: m.progressCh, progressCh: m.progressCh,
} }
@@ -183,12 +203,11 @@ func (m *restoreModel) startRestore() tea.Msg {
if restoreErr != nil { if restoreErr != nil {
m.err = restoreErr m.err = restoreErr
m.progressCh <- -1 m.progressCh <- -1
return } else {
m.progressCh <- -2 // Done signal
} }
return nil
m.progressCh <- -2 // Done signal }
}()
return nil
} }
func (m *restoreModel) restoreFileOrDisk(r io.Reader) error { func (m *restoreModel) restoreFileOrDisk(r io.Reader) error {
@@ -212,7 +231,8 @@ func (m *restoreModel) restoreFileOrDisk(r io.Reader) error {
} }
defer dstFile.Close() defer dstFile.Close()
if _, err = io.Copy(dstFile, gzipReader); err != nil { _, err = io.Copy(dstFile, gzipReader)
if err != nil && err != io.EOF {
return err return err
} }
return dstFile.Sync() return dstFile.Sync()
@@ -264,37 +284,19 @@ func (m *restoreModel) restoreDirectory(r io.Reader) error {
return nil return nil
} }
type restoreProgressReader struct {
reader io.Reader
progressCh chan int64
read int64
}
func (r *restoreProgressReader) Read(p []byte) (int, error) {
n, err := r.reader.Read(p)
if n > 0 {
r.read += int64(n)
r.progressCh <- r.read
}
return n, err
}
func (m *restoreModel) restoreProgressListener() tea.Msg { func (m *restoreModel) restoreProgressListener() tea.Msg {
select { n := <-m.progressCh
case n := <-m.progressCh: if n < 0 {
switch { if n == -1 {
case n == -1: // Ошибка
if m.err == nil { if m.err == nil {
m.err = fmt.Errorf("unknown restoration error") m.err = fmt.Errorf("unknown restoration error")
} }
return restoreErrorMsg{m.err} return restoreErrorMsg{m.err}
case n == -2: // Завершение
return restoreDoneMsg{}
default: // Прогресс
m.processed = n
return restoreProgressMsg(n)
} }
return restoreDoneMsg{}
} }
m.processed = n
return restoreProgressMsg(n)
} }
func (m *restoreModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) { func (m *restoreModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
@@ -309,6 +311,31 @@ func (m *restoreModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
m.progress.Width = msg.Width - 4 m.progress.Width = msg.Width - 4
return m, nil return m, nil
case restoreSourceReadyMsg:
m.totalBytes = msg.totalBytes
if m.totalBytes <= 0 {
m.totalBytes = 1
}
m.isDirectory = msg.isDir // Update directory type, especially for P2P.
// Final check for target directory now that we know the type for sure.
if m.isDirectory && m.target == "" {
wd, err := os.Getwd()
if err != nil {
return m, func() tea.Msg { return restoreErrorMsg{err} }
}
m.target = wd
} else if !m.isDirectory && m.target == "" {
err := fmt.Errorf("target flag -t is required for file/disk restoration")
return m, func() tea.Msg { return restoreErrorMsg{err} }
}
m.statusMessage = "Receiving data..."
return m, tea.Batch(
m.processRestoreCmd(msg.reader),
m.restoreProgressListener,
)
case restoreProgressMsg: case restoreProgressMsg:
progressVal := float64(msg) / float64(m.totalBytes) progressVal := float64(msg) / float64(m.totalBytes)
if progressVal > 1.0 { if progressVal > 1.0 {
@@ -340,6 +367,9 @@ func (m *restoreModel) View() string {
if m.isDirectory { if m.isDirectory {
restoreType = "Directory" restoreType = "Directory"
} }
if m.httpSourceInfo != nil || m.isP2PSource {
restoreType += " network"
}
return fmt.Sprintf("\n✅ %s restoration complete!\n\n"+ return fmt.Sprintf("\n✅ %s restoration complete!\n\n"+
"Source: %s\n"+ "Source: %s\n"+
"Target: %s\n"+ "Target: %s\n"+
@@ -348,12 +378,18 @@ func (m *restoreModel) View() string {
duration.Round(time.Millisecond)) duration.Round(time.Millisecond))
} }
// Show status message before progress starts if m.processed == 0 {
if m.processed == 0 && (m.isNetworkSource || m.statusMessage != "") {
return m.statusMessage + "\n\nPress Ctrl+C to cancel" return m.statusMessage + "\n\nPress Ctrl+C to cancel"
} }
title := fmt.Sprintf("Restoring from %s → %s", m.source, m.target) operation := "Restoring"
if m.httpSourceInfo != nil {
operation = "Downloading"
} else if m.isP2PSource {
operation = "Receiving"
}
title := fmt.Sprintf("%s from %s → %s", operation, m.source, m.target)
progressVal := float64(m.processed) / float64(m.totalBytes) progressVal := float64(m.processed) / float64(m.totalBytes)
progressView := m.progress.ViewAs(progressVal) progressView := m.progress.ViewAs(progressVal)
stats := fmt.Sprintf("%s / %s (%.1f%%)", formatBytes(m.processed), formatBytes(m.totalBytes), progressVal*100) stats := fmt.Sprintf("%s / %s (%.1f%%)", formatBytes(m.processed), formatBytes(m.totalBytes), progressVal*100)
@@ -385,7 +421,7 @@ func isBlockDevice(path string) bool {
func init() { func init() {
rootCmd.AddCommand(restoreCmd) rootCmd.AddCommand(restoreCmd)
restoreCmd.Flags().StringVarP(&restoreSource, "source", "s", "", "Source archive or network address (e.g., /path/to/archive.tar.gz or 0.0.0.0:8080) (required)") restoreCmd.Flags().StringVarP(&restoreSource, "source", "s", "", "Source archive, P2P address (e.g., 0.0.0.0:8080), or HTTP URL (e.g., 127.0.0.1:8080/backup.gz) (required)")
restoreCmd.Flags().StringVarP(&restoreTarget, "target", "t", "", "Target file, disk, or directory. Required for network sources.") restoreCmd.Flags().StringVarP(&restoreTarget, "target", "t", "", "Target file, disk, or directory. Required for non-directory restores.")
restoreCmd.MarkFlagRequired("source") restoreCmd.MarkFlagRequired("source")
} }
+96
View File
@@ -0,0 +1,96 @@
package cmd
import (
"fmt"
"io"
"log"
"net/http"
"os"
"path/filepath"
"github.com/gorilla/mux"
"github.com/spf13/cobra"
)
var (
serveAddress string
serveDirectory string
)
var serveCmd = &cobra.Command{
Use: "serve",
Short: "Run a backup server",
Long: "Run an HTTP server to upload and download backups.",
Run: func(cmd *cobra.Command, args []string) {
if serveDirectory == "" {
home, err := os.UserHomeDir()
if err != nil {
log.Fatalf("Failed to get user home directory: %v", err)
}
serveDirectory = filepath.Join(home, "backups")
}
if err := os.MkdirAll(serveDirectory, 0755); err != nil {
log.Fatalf("Failed to create backup directory: %v", err)
}
r := mux.NewRouter()
r.HandleFunc("/backup/{filename}", uploadHandler).Methods("POST")
r.HandleFunc("/backup/{filename}", downloadHandler).Methods("GET")
log.Printf("Starting server on %s", serveAddress)
log.Printf("Using backup directory: %s", serveDirectory)
if err := http.ListenAndServe(serveAddress, r); err != nil {
log.Fatalf("Server failed: %v", err)
}
},
}
func uploadHandler(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
filename := vars["filename"]
filePath := filepath.Join(serveDirectory, filename)
file, err := os.Create(filePath)
if err != nil {
http.Error(w, "Failed to create file", http.StatusInternalServerError)
log.Printf("Error creating file %s: %v", filename, err)
return
}
defer file.Close()
_, err = io.Copy(file, r.Body)
if err != nil {
http.Error(w, "Failed to write to file", http.StatusInternalServerError)
log.Printf("Error writing to file %s: %v", filename, err)
return
}
w.WriteHeader(http.StatusOK)
fmt.Fprintf(w, "File %s uploaded successfully.", filename)
log.Printf("Uploaded %s", filename)
}
func downloadHandler(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
filename := vars["filename"]
filePath := filepath.Join(serveDirectory, filename)
// Проверяем, существует ли файл, перед отправкой
if _, err := os.Stat(filePath); os.IsNotExist(err) {
http.NotFound(w, r)
log.Printf("File not found: %s", filename)
return
}
// http.ServeFile - это идиоматический способ отправки файлов в Go.
// Он автоматически устанавливает Content-Type, Content-Length и другие заголовки.
http.ServeFile(w, r, filePath)
log.Printf("Downloaded %s", filename)
}
func init() {
rootCmd.AddCommand(serveCmd)
serveCmd.Flags().StringVarP(&serveAddress, "address", "a", "localhost:8080", "Address and port for the server")
serveCmd.Flags().StringVarP(&serveDirectory, "directory", "d", "", "Directory to store backups (defaults to ~/backups)")
}
+80
View File
@@ -2,6 +2,8 @@ package cmd
import ( import (
"fmt" "fmt"
"io"
"net"
"strings" "strings"
) )
@@ -22,3 +24,81 @@ func formatBytes(b int64) string {
} }
return fmt.Sprintf("%.1f %ciB", float64(b)/float64(div), "KMGTPE"[exp]) return fmt.Sprintf("%.1f %ciB", float64(b)/float64(div), "KMGTPE"[exp])
} }
// ServerTargetInfo holds the parsed information from a server target string.
type ServerTargetInfo struct {
// Full HTTP URL for the request, e.g., http://127.0.0.1:8080/backup/mybackup.tar.gz
URL string
// The address of the server, e.g., 127.0.0.1:8080
Address string
// The filename for the backup, e.g., mybackup.tar.gz
Filename string
}
// parseServerTarget parses a target string like `127.0.0.1:8080/backup.gz`.
// It returns a struct with the full URL and filename, or an error if the format is invalid.
func parseServerTarget(target string) (*ServerTargetInfo, bool) {
if !strings.Contains(target, "/") || !strings.Contains(target, ":") {
return nil, false
}
parts := strings.SplitN(target, "/", 2)
if len(parts) != 2 {
return nil, false // Invalid format
}
address := parts[0]
filename := parts[1]
if filename == "" {
return nil, false // Filename cannot be empty
}
// Validate that the first part is a host:port
host, port, err := net.SplitHostPort(address)
if err != nil {
return nil, false // Not a valid host:port
}
if host == "" || port == "" {
return nil, false
}
// It looks like a valid server target.
fullURL := fmt.Sprintf("http://%s/backup/%s", address, filename)
return &ServerTargetInfo{
URL: fullURL,
Address: address,
Filename: filename,
}, true
}
// progressTracker реализует io.Reader и io.Writer для отслеживания прогресса
type progressTracker struct {
Reader io.Reader
Writer io.Writer
progressCh chan int64
// Для restore, где io.Copy может вызываться много раз (в tar),
// нам нужно отслеживать общий прогресс.
processed int64
}
// Write отслеживает прогресс записи (для create)
func (pt *progressTracker) Write(p []byte) (int, error) {
n, err := pt.Writer.Write(p)
if n > 0 {
pt.progressCh <- int64(n) // Отправляем дельту
}
return n, err
}
// Read отслеживает прогресс чтения (для restore)
func (pt *progressTracker) Read(p []byte) (int, error) {
n, err := pt.Reader.Read(p)
if n > 0 {
pt.processed += int64(n)
pt.progressCh <- pt.processed // Отправляем общий обработанный объем
}
return n, err
}
+2 -1
View File
@@ -6,7 +6,9 @@ require (
github.com/charmbracelet/bubbles v0.21.0 github.com/charmbracelet/bubbles v0.21.0
github.com/charmbracelet/bubbletea v1.3.4 github.com/charmbracelet/bubbletea v1.3.4
github.com/charmbracelet/lipgloss v1.1.0 github.com/charmbracelet/lipgloss v1.1.0
github.com/gorilla/mux v1.8.1
github.com/spf13/cobra v1.9.1 github.com/spf13/cobra v1.9.1
golang.org/x/sys v0.30.0
) )
require ( require (
@@ -29,6 +31,5 @@ require (
github.com/spf13/pflag v1.0.6 // indirect github.com/spf13/pflag v1.0.6 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
golang.org/x/sync v0.11.0 // indirect golang.org/x/sync v0.11.0 // indirect
golang.org/x/sys v0.30.0 // indirect
golang.org/x/text v0.3.8 // indirect golang.org/x/text v0.3.8 // indirect
) )