Skip to content

Commit

Permalink
chore: optimize docs
Browse files Browse the repository at this point in the history
Signed-off-by: Jan Lauber <jan.lauber@protonmail.ch>
  • Loading branch information
janlauber committed May 8, 2024
1 parent 4c00be7 commit 3231346
Show file tree
Hide file tree
Showing 7 changed files with 161 additions and 55 deletions.
75 changes: 49 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,44 @@
# MySQL Backup Docker Container

This Docker container is designed to backup MySQL databases and upload the backups to Google Cloud Storage. It also handles the management of a Cloud SQL Proxy connection to securely connect to Google Cloud SQL instances without having to expose them publicly.
This Docker container is designed to backup MySQL databases and upload the backups to Google Cloud Storage. It also handles the management of a Cloud SQL Proxy connection to securely connect to Google Cloud SQL instances without exposing them publicly.

## Features

- **Database Backups**: Supports backing up multiple databases configured via environment variables.
- **Google Cloud Storage**: Uploads database backups directly to a configured GCS bucket.
- **Cloud SQL Proxy Management**: Handles the lifecycle of a Cloud SQL Proxy connection to secure database connections.
- **Google Cloud Storage**: Automatically uploads database backups to a specified GCS bucket.
- **Cloud SQL Proxy Management**: Manages the lifecycle of a Cloud SQL Proxy connection to secure database access.

## Prerequisites

- A Google Cloud account and a configured GCS bucket.
- A Google Cloud account with a configured GCS bucket.
- Access to a Google Cloud SQL instance or any MySQL server.
- Docker installed on your machine or Kubernetes cluster.

## Configuration

### Environment Variables

This container uses the following environment variables:
This container relies on several environment variables for operation:

- `MYSQL_HOST`: The hostname of the MySQL server.
- `MYSQL_USER`: The username for the MySQL database.
- `MYSQL_PASSWORD`: The password for the MySQL database (use Kubernetes secrets for Kubernetes deployments).
- `MYSQL_DATABASES`: Comma-separated list of databases to backup.
- `GCS_BUCKET`: The Google Cloud Storage bucket where backups will be stored.
- `PROXY_PORT`: The port on which the MySQL server or proxy is accessible (default is 3306).
- `PROXY_QUIT_URL`: URL to send the quit command to Cloud SQL Proxy.
- `MYSQL_HOST` (**Required**): The hostname of the MySQL server.
- `MYSQL_USER` (**Required**): The username for the MySQL database.
- `MYSQL_PASSWORD` (**Required**): The password for the MySQL database (recommended to use Kubernetes secrets for Kubernetes deployments).
- `MYSQL_DATABASES` (**Required**): Comma-separated list of databases to backup.
- `GCS_BUCKET` (**Required**): The Google Cloud Storage bucket for storing backups.
- `GCS_BUCKET_DIR` (**Required**): The directory within the GCS bucket to store backups.
- `PROXY_PORT` (**Required**): The port on which the MySQL server or proxy is accessible (default 3306).
- `PROXY_QUIT_URL` (**Required**): URL to send the quit command to the Cloud SQL Proxy.

### Volumes

- `/backup`: This directory is where the database dumps are temporarily stored before being uploaded to GCS.
- `/backup`: The directory where database dumps are temporarily stored before being uploaded to GCS.

## Using the Docker Container
## Usage

### Running Locally

To run the Docker container locally:

```sh
docker run -e MYSQL_HOST='your-mysql-host' -e MYSQL_USER='user' -e MYSQL_PASSWORD='password' \
-e MYSQL_DATABASES='db1,db2' -e GCS_BUCKET='your-gcs-bucket' \
Expand All @@ -46,32 +49,52 @@ your-docker-image

### Deployment in Kubernetes

You can deploy this container in a Kubernetes cluster as part of a CronJob. Refer to the included `kubernetes-cronjob.yaml` file for a sample deployment configuration.
Deploy this container in a Kubernetes cluster as part of a CronJob. Refer to the [deployment](./deployment/) directory for a Kubernetes deployment example.

```sh
kubectl apply -k deployment/
```

### GCP Service Account & IAM Role

Set up a service account in GCP and assign necessary roles for Cloud SQL and GCS access:

```sh
gcloud iam service-accounts create cloudsql-backup-sa --display-name "Cloud SQL Backup Service Account"

gcloud projects add-iam-policy-binding your-project-id \
--member serviceAccount:cloudsql-backup-sa@your-project-id.iam.gserviceaccount.com \
--role roles/cloudsql.client

gcloud projects add-iam-policy-binding your-project-id \
--member serviceAccount:cloudsql-backup-sa@your-project-id.iam.gserviceaccount.com \
--role roles/storage.objectAdmin
```

Bind this service account to your Kubernetes service account:

```sh
gcloud iam service-accounts add-iam-policy-binding \
--role roles/iam.workloadIdentityUser \
--member "serviceAccount:your-project-id.svc.id.goog[your-namespace/your-service-account]"
```

## Building the Docker Image

To build this image from the Dockerfile:
To build the image from the Dockerfile:

```sh
docker build -t your-custom-tag .
```

## Contributing

Contributions to this project are welcome! Please fork the repository and submit a pull request with your changes or improvements.
We welcome contributions! Please fork the repository and submit pull requests with your changes or improvements.

## License

Specify the license under which the project is available. Common licenses for open source projects include MIT, Apache 2.0, etc.

## Contact Information

For help or issues, please submit an issue to the project's GitHub repository or contact the maintainer at `support@natron.io`.


### Notes on the README

- **Detailed Instructions**: The README provides instructions on how to use the Docker image, including environment variables, volume mappings, and running the container.
- **Example Commands**: Commands for running the container both locally and within a Kubernetes environment are given to ease the usage.
- **Contribution Guidelines**: Encouraging contributions helps in the growth and maintenance of the project.
- **License**: It's important to specify a license if the project is open source. This governs how the project can be used and modified by others.
For help or issues related to this project, please submit an issue to our GitHub repository or contact the maintainer at `support@natron.io`.
29 changes: 0 additions & 29 deletions cronjob.yaml

This file was deleted.

84 changes: 84 additions & 0 deletions deployment/cronjob.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
apiVersion: batch/v1
kind: CronJob
metadata:
name: backup-gcp-mysql
namespace: mysql-backups
spec:
ttlSecondsAfterFinished: null
schedule: "0 5 * * *"
concurrencyPolicy: Forbid
suspend: false
jobTemplate:
metadata:
labels:
app: backup-gcp-mysql
spec:
parallelism: 1
completions: 1
backoffLimit: 3
template:
metadata:
labels:
app: backup-gcp-mysql
spec:
containers:
- name: backup-job
image: ghcr.io/natrontech/gcp-mysql-backup:latest
env:
- name: MYSQL_HOST
value: "127.0.0.1"
- name: PROXY_PORT
value: "3306"
- name: MYSQL_USER
valueFrom:
secretKeyRef:
name: backup-gcp-mysql
key: MYSQL_USER
- name: MYSQL_PASSWORD
valueFrom:
secretKeyRef:
name: backup-gcp-mysql
key: MYSQL_PASSWORD
- name: MYSQL_DATABASES
value: "db1, db2, db3" # Add more databases separated by comma
- name: GCP_BUCKET
value: "backups"
- name: GCP_BUCKET_DIR
value: "mysql"
- name: PROXY_QUIT_URL
value: "http://localhost:9091/quitquitquit"
- name: cloud-sql-proxy
image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.11.0
args:
- '--exit-zero-on-sigterm'
- '--quitquitquit'
- '--port=3306'
- project-id:region:instance-name # Replace with your project-id, region and instance name
resources:
limits:
cpu: 500m
memory: 1Gi
requests:
cpu: 250m
memory: 256Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: Always
securityContext:
runAsNonRoot: true
restartPolicy: OnFailure
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
serviceAccountName: backup-gcp-mysql
serviceAccount: backup-gcp-mysql
automountServiceAccountToken: true
securityContext:
runAsUser: 1001
runAsGroup: 3000
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
nodeSelector:
cloud.google.com/gke-nodepool: system-pool
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 5
8 changes: 8 additions & 0 deletions deployment/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mysql-backups
resources:
- ns.yaml
- serviceaccount.yaml
- secret.yaml
- cronjob.yaml
4 changes: 4 additions & 0 deletions deployment/ns.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
apiVersion: v1
kind: Namespace
metadata:
name: mysql-backups
9 changes: 9 additions & 0 deletions deployment/secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: v1
kind: Secret
metadata:
name: backup-gcp-mysql
namespace: mysql-backups
data:
MYSQL_PASSWORD: <base64 encoded password>
MYSQL_USER: <base64 encoded username>
type: Opaque
7 changes: 7 additions & 0 deletions deployment/serviceaccount.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: backup-gcp-mysql
namespace: mysql-backups
annotations:
iam.gke.io/gcp-service-account: "your-gcp-service-account@your-project-id.iam.gserviceaccount.com" # Replace with your GCP service account email

0 comments on commit 3231346

Please sign in to comment.