Differences

This shows you the differences between two versions of the page.

--- porting_an_lxc_to_docker [2024/08/11 03:54] – [1. Setting Up Dependencies] oso
+++ porting_an_lxc_to_docker [2024/10/17 21:42] (current) – external edit 127.0.0.1
@@ Line 37: / Line 37: @@
 The process of containerizing the Flask application involved several key steps, including setting up the necessary dependencies, creating a Dockerfile, and configuring the `docker-compose.yml` file.
-====== v1 - Using Clean Docker Images for Python Apps and MariaDB ======
+====== Using Clean Docker Images for Python Apps and MariaDB ======
 When transitioning from LXC containers to Docker, it's often more efficient and cleaner to use base Docker images for your applications rather than attempting to convert existing LXC containers. This approach offers several advantages:
@@ Line 47: / Line 47: @@
 ===== General Outline for Container Creation =====
-Here’s a general outline of the steps to create Docker containers for a Python application and a MariaDB database.
+Here’s a general outline of the steps to create Docker containers for a Python application writing to a MariaDB database.
-==== 1. Create Dockerfile for Python App ====
+==== 1. Setting Up Dependencies ====
-First, create a `Dockerfile` for your Python application. Use a base image like Debian or Alpine Ubuntu.
+Before containerizing the application, it was crucial to define the required dependencies in a `requirements.txt` file. The Flask application used the following dependencies:
-  * Create a `Dockerfile`:
 <code>
+Flask==2.1.2
+mariadb==1.1.4
+</code>
+These versions ensured compatibility and stability, avoiding issues related to breaking changes in more recent versions.
+==== 2. Creating the Dockerfile ====
+The next step was to create a `Dockerfile` to define the environment for the Flask application. The Dockerfile included the following instructions:
+<code dockerfile>
 FROM python:3.9-slim
+# Install MariaDB Connector/C library and other dependencies
+RUN apt-get update && apt-get install -y \
+    libmariadb3 \
+    libmariadb-dev \
+    gcc \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
 WORKDIR /app
@@ Line 63: / Line 80: @@
 COPY . .
-CMD ["python", "app.py"]
+CMD ["python", "main.py"]
 </code>
-  * Build the Docker image:
+**Explanation**:
-<code>
+  * `FROM python:3.9-slim`: This uses a lightweight Python 3.9 image as the base.
-docker build -t my-python-app .
+  * **Explanation of MariaDB Connector/C and Dependencies**
+<code dockerfile>
+# Install MariaDB Connector/C library and other dependencies
+RUN apt-get update && apt-get install -y \
+    libmariadb3 \
+    libmariadb-dev \
+    gcc \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
 </code>
+  * **`RUN apt-get update && apt-get install -y \`**:
+    - This command updates the package list on the Debian-based slim image and installs the specified packages.
+  * **`libmariadb3`**:
+    - **What it is**: This is the shared library for MariaDB Connector/C, which is a lightweight client library to connect your Flask application to a MariaDB database.
+    - **Why it’s needed**: If your Flask app interacts with a MariaDB database, this library allows it to establish that connection.
+  * **`libmariadb-dev`**:
+    - **What it is**: This is the development package for MariaDB Connector/C. It includes the headers and static libraries needed for compiling programs that use MariaDB.
+    - **Why it’s needed**: If you have Python packages in `requirements.txt` that need to compile C extensions (for example, `mysqlclient`), they require these development files to build correctly.
+  * **`gcc`**:
+    - **What it is**: The GNU Compiler Collection, a compiler system that supports various programming languages.
+    - **Why it’s needed**: This is required to compile any C extensions or Python packages that need compilation, particularly those that depend on the MariaDB libraries mentioned above.
+  * **`&& apt-get clean && rm -rf /var/lib/apt/lists/*`**:
+    - **What it does**: This cleans up the package lists and removes any cached data from `apt-get`, which reduces the size of the Docker image by removing unnecessary files.
+  * `WORKDIR /app`: Sets the working directory inside the container.
+  * `COPY requirements.txt requirements.txt`: Copies the `requirements.txt` file into the container.
+  * `RUN pip install -r requirements.txt`: Installs the required Python packages.
+  * `COPY . .`: Copies all the application files into the container.
-==== 2. Create Dockerfile for MariaDB (optional) ====
+==== 3. Configuring Docker-Compose ====
-Next, create a `Dockerfile` for MariaDB or use the official MariaDB image.
+To orchestrate the Flask application in a multi-container environment, a `docker-compose.yml` file was created with the following content:
-  * Use the official MariaDB image in your `docker-compose.yml`:
 <code>
 version: '3.1'
 services:
-  db:
-	image: mariadb
-	restart: always
-	environment:
-	  MYSQL_ROOT_PASSWORD: example
-	  MYSQL_DATABASE: mydb
-	  MYSQL_USER: user
-	  MYSQL_PASSWORD: password
   app:
-	build: .
+    build: .
-	ports:
+    ports:
-	  - "5000:5000"
+      - "5000:5000"
-	depends_on:
+    environment:
-	  - db
+      - FLASK_ENV=development
+      - API_TOKEN=bXktbG9uZy***LongAssBase64String***cmluZw==
+      - DB_HOST=api.facundoitest.space
+      - DB_PASSWORD=***myPassword***
 </code>
-==== 3. Configure Docker Compose ====
+**Explanation**:
-Use Docker Compose to manage both containers. Create a `docker-compose.yml` file.
+  * `services`: Defines the application service.
+  * `build: .`: Tells Docker Compose to build the image using the Dockerfile in the current directory.
+  * `ports: "5000:5000"`: Maps port 5000 on the host to port 5000 in the container.
+==== 4. Building and Running the Container ====
+After setting up the Dockerfile and `docker-compose.yml`, the following commands were used to build and run the Flask application in a Docker container:
-  * Example `docker-compose.yml`:
 <code>
-version: '3.1'
+docker-compose build
+docker-compose up -d
+</code>
-services:
+**Explanation**:
-  app:
+  * `docker-compose build`: Builds the Docker image based on the instructions in the Dockerfile.
-	build: .
+  * `docker-compose up -d`: Starts the container in detached mode.
-	ports:
-	  - "5000:5000"
-	depends_on:
-	  - db
-  db:
+==== 5. App code with environment variables ====
-	image: mariadb
-	restart: always
-	environment:
-	  MYSQL_ROOT_PASSWORD: example
-	  MYSQL_DATABASE: mydb
-	  MYSQL_USER: user
-	  MYSQL_PASSWORD: password
-</code>
-==== 4. Deploy Containers ====
+<code python>
-Deploy your containers using Docker Compose.
+from flask import Flask, request, jsonify, abort, g
+from datetime import datetime
+import mariadb
+import re
+import os
-  * Run the containers:
+# Get passwords and tokens from environment variables
-<code>
+API_TOKEN = os.environ.get('API_TOKEN')
-docker-compose up -d
+DB_PASSWORD = os.environ.get('DB_PASSWORD')
-</code>
+DB_HOST = os.environ.get('DB_HOST')
-==== 5. Test and Verify ====
+app = Flask(__name__)
-Ensure that both containers are running and communicating correctly.
-  * Check container status:
+def get_db():
-<code>
+    if 'db' not in g:
-docker ps
+        g.db = mariadb.connect(
-</code>
+            host=DB_HOST,
+            port=3306,
+            user="facundo",
+            password=DB_PASSWORD,
+            database="VeeamReports"
+        )
+    return g.db
-  * Verify the application is accessible:
+@app.before_request
-    Open your browser and navigate to `http://localhost:5000`.
+def before_request():
+    get_db()
-===== Conclusion =====
+@app.teardown_request
+def teardown_request(exception=None):
+    db = g.pop('db', None)
+    if db is not None:
+        db.close()
-Using clean Docker images for your Python applications and databases provides a more maintainable, scalable, and universally compatible solution compared to converting LXC containers. This approach aligns with modern best practices in containerization and microservices architecture.
+@app.route('/upload', methods=['POST'])
+def upload_file():
+    db = get_db()  # Get the database connection
+    cursor = db.cursor()  # Now 'db' should be defined
+    # Check for Authorization header
+    if 'Authorization' not in request.headers:
+        abort(401)  # Unauthorized
-===== V2 =====
+    # Check if the token is correct
+    token = request.headers['Authorization']
+    if token != API_TOKEN:
+        abort(403)  # Forbidden
-==== 1. Setting Up Dependencies ====
+    data = request.get_json()  # Get JSON data from request
-Before containerizing the application, it was crucial to define the required dependencies in a `requirements.txt` file. The Flask application used the following dependencies:
+    if data is None:
+        return jsonify({'error': 'No JSON received.'}), 400
+    # ---------- The application itself ---------
+    # Extract hostname from data
+    hostname = data['hostname']
+    # Create a new table for this hostname if it doesn't exist
+    cursor.execute(f"CREATE TABLE IF NOT EXISTS `{hostname}` (id INT AUTO_INCREMENT PRIMARY KEY, creationtime VARCHAR(255), vmname VARCHAR(255), type VARCHAR(255), result VARCHAR(255))")
+    # Delete all rows from the table
+    cursor.execute(f"DELETE FROM `{hostname}`")
+    for restorePoint in data['restorePoints']:
+        creationtime = restorePoint['creationtime']
+        # Check if creationtime contains Date
+        if "Date" in creationtime:
+            # Extract the integer from the creationtime string
+            match = re.search(r'\((\d+)\)', creationtime)
+            if match:
+                creationtime = int(match.group(1))  # Convert to integer type
+        # Check the type and replace it if necessary
+        type = str(restorePoint['type'])  # wtf
+        if str(restorePoint['type']) == '0':
+            type = 'Full'
+        elif str(restorePoint['type']) == '2':
+            type = 'Incremental'
+        elif str(restorePoint['type']) == '4':
+            type = 'Snapshot'
+        # Check the type and replace the result if necessary
+        if str(restorePoint['type']) in ["TieringJob", "BackupJob"]:
+            if str(restorePoint['result']) == '0':
+                result = 'Success'
+            elif str(restorePoint['result']) == '1':
+                result = 'Warn'
+            elif str(restorePoint['result']) == '2':
+                result = 'Fail'
+        else:
+            result = restorePoint['result']
+        query = f"INSERT INTO `{hostname}` (creationtime, vmname, type, result) VALUES (%s, %s, %s, %s)"
+        values = (
+            creationtime,
+            restorePoint['vmname'],
+            type,
+            result
+        )
+        cursor.execute(query, values)
+    db.commit()  # Commit the transaction
+    return jsonify({'message': 'Data saved successfully.'}), 200
+if __name__ == '__main__':
+    app.run(host='0.0.0.0', port=5000, debug=True)
+</code>
+==== Troubleshooting MariaDB import ====
 <code>
-Flask==2.1.2
+#5 11.31       OSError: mariadb_config not found.
-mariadb==1.1.4
+#5 11.31
+#5 11.31       This error typically indicates that MariaDB Connector/C, a dependency which
+#5 11.31       must be preinstalled, is not found.
+#5 11.31       If MariaDB Connector/C is not installed, see installation instructions
+#5 11.31       If MariaDB Connector/C is installed, either set the environment variable
+#5 11.31       MARIADB_CONFIG or edit the configuration file 'site.cfg' to set the
+#5 11.31        'mariadb_config' option to the file location of the mariadb_config utility.
+#5 11.31
+#5 11.31       [end of output]
 </code>
-These versions ensured compatibility and stability, avoiding issues related to breaking changes in more recent versions.
+The error you're seeing indicates that the `mariadb` Python package requires the MariaDB Connector/C library to be preinstalled, and it's not available in the Docker image you're using. You need to install this library before installing your Python requirements.
-==== 2. Creating the Dockerfile ====
+Here's how you can modify your Dockerfile to install the MariaDB Connector/C library before installing the Python dependencies:
-The next step was to create a `Dockerfile` to define the environment for the Flask application. The Dockerfile included the following instructions:
-<code>
+=== Updated Dockerfile ===
+<code dockerfile>
 FROM python:3.9-slim
+# Install MariaDB Connector/C library and other dependencies
+RUN apt-get update && apt-get install -y \
+    libmariadb3 \
+    libmariadb-dev \
+    gcc \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
 WORKDIR /app
@@ Line 170: / Line 300: @@
 COPY . .
-CMD ["gunicorn", "--bind", "0.0.0.0:5000", "main:app"]
+CMD ["python", "main.py"]
 </code>
-**Explanation**:
+=== Explanation ===
-  * `FROM python:3.9-slim`: This uses a lightweight Python 3.9 image as the base.
-  * `WORKDIR /app`: Sets the working directory inside the container.
-  * `COPY requirements.txt requirements.txt`: Copies the `requirements.txt` file into the container.
-  * `RUN pip install -r requirements.txt`: Installs the required Python packages.
-  * `COPY . .`: Copies all the application files into the container.
-  * `CMD ["gunicorn", "--bind", "0.0.0.0:5000", "main:app"]`: Specifies the command to run the application using Gunicorn.
-==== 3. Configuring Docker-Compose ====
+  * **Update and install dependencies**: The `RUN apt-get update && apt-get install -y` command installs the required MariaDB Connector/C library (`libmariadb3` and `libmariadb-dev`) and a C compiler (`gcc`).
-To orchestrate the Flask application in a multi-container environment, a `docker-compose.yml` file was created with the following content:
+  * **Clean up**: The `apt-get clean && rm -rf /var/lib/apt/lists/*` commands clean up the apt cache to reduce the image size.
-<code>
+Now, try building your Docker image again with the updated Dockerfile:
-version: '3'
+<code bash>
+docker-compose up --build
+</code>
-services:
+This should resolve the issue with the missing `mariadb_config` and allow the `mariadb` Python package to install correctly.
-  app:
-    build: .
-    ports:
-      - "5000:5000"
-    environment:
-      API_TOKEN: ${API_TOKEN}
-    networks:
-      - app-network
-networks:
+==== Troubleshooting Werkzeug ====
-  app-network:
+<code>
+app-flask-app-1  | ImportError: cannot import name 'url_quote' from 'werkzeug.urls' (/usr/local/lib/python3.9/site-packages/werkzeug/urls.py)
+app-flask-app-1 exited with code 1
 </code>
-**Explanation**:
+The error you're encountering is due to an incompatibility between Flask and Werkzeug versions. Flask 2.1.2 may not be compatible with the latest version of Werkzeug.
-  * `services`: Defines the application service.
-  * `build: .`: Tells Docker Compose to build the image using the Dockerfile in the current directory.
-  * `ports: "5000:5000"`: Maps port 5000 on the host to port 5000 in the container.
-  * `environment: API_TOKEN: ${API_TOKEN}`: Passes the `API_TOKEN` environment variable to the container.
-  * `networks`: Creates and attaches the container to a custom network, `app-network`.
-==== 4. Building and Running the Container ====
+To fix this, you can pin the Werkzeug version to one that is compatible with Flask 2.1.2. Based on the error, pinning Werkzeug to version 2.0.3 should resolve the issue.
-After setting up the Dockerfile and `docker-compose.yml`, the following commands were used to build and run the Flask application in a Docker container:
+=== Updated requirements.txt ===
 <code>
-docker-compose build
+Flask==2.1.2
-docker-compose up -d
+mariadb==1.1.4
+Werkzeug==2.0.3
 </code>
-**Explanation**:
+=== Dockerfile (unchanged from previous update) ===
-  * `docker-compose build`: Builds the Docker image based on the instructions in the Dockerfile.
-  * `docker-compose up -d`: Starts the container in detached mode.
-==== 5. Troubleshooting and Final Adjustments ====
+<code Dockerfile>
-During the initial deployment, the following issues were encountered:
+FROM python:3.9-slim
-  * **Werkzeug Compatibility**: The application initially failed due to an incompatibility with `Werkzeug` version 2.2.3, which was resolved by pinning the version to 2.0.3 in `requirements.txt`.
-  * **Missing Imports**: The application threw an error due to a missing `os` module import, which was quickly added to resolve the issue.
-By following these steps, the Flask application was successfully containerized and deployed using Docker, providing a scalable and consistent environment for running the application.
+# Install MariaDB Connector/C library and other dependencies
+RUN apt-get update && apt-get install -y \
+    libmariadb3 \
+    libmariadb-dev \
+    gcc \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
+WORKDIR /app
+COPY requirements.txt requirements.txt
+RUN pip install -r requirements.txt
+COPY . .
+CMD ["python", "main.py"]
+</code>
+=== Steps ===
+  - **Update your `requirements.txt`** as shown above.
+  - **Rebuild the Docker image** with the updated `requirements.txt`:
+  docker-compose up --build
+This should resolve the incompatibility issue between Flask and Werkzeug and allow your Flask app to start without errors.
 ===== Conclusion =====
 Porting from LXC to Docker is not a trivial task due to the differences in how these containers are designed and managed. It's essential to review the services, configurations, and workflows involved before starting the migration to avoid unexpected issues and ensure that your applications run smoothly in Docker.
@@ Line 236: / Line 372: @@
   * LXC documentation: https://linuxcontainers.org/lxc/introduction/
+https://docs.facundoitest.space/doku.php?id=crear_container_con_apps_en_python_para_aci