diff --git a/server/README.md b/server/README.md index 35bc430..67136fe 100644 --- a/server/README.md +++ b/server/README.md @@ -1,13 +1,26 @@ # Sensor Readouts -A Flask web application for managing and visualizing sensor data. It provides a REST API for creating sensors and storing readouts, along with a frontend dashboard that displays real-time charts and tabular data. +Two Flask services with separate responsibilities and separate databases: + +- **main app** (`app.py`) — all user-facing views: login, home, API docs and + the `/sensors` dashboard. It owns the **users** database (`main_app.db`) and + proxies data requests to the sensors API. +- **sensors app** (`sensors_app.py`) — API only. It owns the **sensors** + database (`sensors.db`, tables `sensor` and `sensor_readout`) and has no + knowledge of users. Its API is open to every user on the local machine. + +``` +browser ──> main app :8000 ──HTTP──> sensors API :8001 + │ │ + main_app.db sensors.db +``` ## Tech Stack - **Python 3.13+** - **Flask 3** — web framework - **SQLAlchemy** (via Flask-SQLAlchemy) — ORM -- **SQLite** — database +- **SQLite** — databases - **Alembic** — database migrations - **Chart.js** — frontend charting library @@ -15,18 +28,31 @@ A Flask web application for managing and visualizing sensor data. It provides a ``` server/ -├── app.py # Flask application factory and API routes -├── models.py # SQLAlchemy models (Sensor, SensorReadout) -├── seed.py # Database seeder with sample data -├── requirements.txt # Python dependencies -├── data.db # SQLite database (auto-created) -├── alembic.ini # Alembic configuration -├── alembic/ # Migration scripts -│ ├── env.py -│ └── versions/ +├── app.py # Main app: views + API proxy +├── sensors_app.py # Sensors app: API + sensors database +├── sensors_client.py # HTTP client used by the main app +├── auth.py # Flask-Login setup +├── config.py # Config (databases, secret key, sensors URL) +├── models.py # Sensor, SensorReadout (sensors DB) +├── user_models.py # User (main_app.db) +├── seed.py # Sample sensor data +├── migrate_legacy_db.py # Import old combined data.db into split DBs +├── run.sh # Development launcher (main/sensors + port) +├── run-prod.sh # Production launcher (gunicorn) +├── gunicorn.conf.py # Gunicorn config (bind from BIND/PORT) +├── alembic.ini # Sensors DB migrations +├── alembic/ # Sensors DB migration scripts +├── alembic_users.ini # Users DB migrations +├── alembic_users/ # Users DB migration scripts +├── sensors.db # Sensors database (sensors app) +├── main_app.db # Users database (main app) +├── data.db # Legacy combined DB (not used by default) +├── requirements.txt └── templates/ - ├── sensors.html # Dashboard with chart and table - └── docs.html # API documentation page + ├── home.html + ├── login.html + ├── docs.html + └── sensors.html ``` ## Setup @@ -39,20 +65,50 @@ source .venv/bin/activate # Install dependencies pip install -r requirements.txt -# Run database migrations +# Sensors database alembic upgrade head -# (Optional) Seed the database with sample data +# Users database +alembic -c alembic_users.ini upgrade head + +# (Optional) Sample sensor data python seed.py ``` +The `sensors.db` and `main_app.db` files are created by the migrations and are +ignored by git (`*.db`). + +### Migrating the old combined database + +Previous versions stored users, sensors and readouts in `data.db`. The new +layout uses `sensors.db` and `main_app.db`. To copy the old rows (the legacy file +is only read, never modified): + +```bash +alembic upgrade head +alembic -c alembic_users.ini upgrade head +python migrate_legacy_db.py +``` + +For custom paths use `--legacy`, `--sensors` and `--main-app`, for example: + +```bash +python migrate_legacy_db.py --legacy old.db --sensors /data/sensors.db \ + --main-app /data/main_app.db +``` + +After verifying the new databases you can archive or remove `data.db`. + ## Users -There is no public registration. All accounts are created by an administrator. -Pages (`/`, `/home`, `/sensors`, `/api/docs`) and dashboard data endpoints -(`GET /api/sensors/:id/types`, `/readouts`, `/readouts/hourly`) require login. -`GET /api/sensors` (list, needed by devices to resolve name to id) and -`POST /api/*` stay open so devices can push data without a browser session. +User accounts live only in `main_app.db` (main app). There is no public +registration; accounts are created by an administrator. + +Main app pages (`/`, `/home`, `/api/docs`, `/sensors`) and dashboard read +endpoints (`GET /api/sensors/:id/types`, `/readouts`, `/readouts/hourly`) +require login. Device endpoints (`GET/POST /api/sensors`, +`POST /api/sensors/:id/readouts`) stay open so devices can push data without a +browser session. Set a secret key for sessions (required in production): @@ -60,15 +116,12 @@ Set a secret key for sessions (required in production): export SECRET_KEY='long-random-value' ``` +If `SECRET_KEY` is unset, the main app falls back to `.secret_key` in this +directory (created automatically by `run-prod.sh`). + ### First admin -The database starts with an empty `user` table. Create the first admin -with the CLI command (flags, env vars, or interactive prompts): - ```bash -# Run database migrations first -alembic upgrade head - # Option 1: flags flask --app app create-admin --username admin --password 'secret' @@ -79,47 +132,95 @@ ADMIN_USERNAME=admin ADMIN_PASSWORD=secret flask --app app create-admin flask --app app create-admin ``` -Running `create-admin` for an existing username resets its password and -grants `is_admin=True`. After that log in at `/login` — you land on `/home`. +Running `create-admin` for an existing username resets its password and grants +`is_admin=True`. Log in at `/login`; you land on `/home`. ## Running +The first argument selects the app (`main` or `sensors`), the second the port. +The main app (`:8000`) listens on all interfaces; the sensors API (`:8001`) +listens on `127.0.0.1` by default and is open to every local user without +authentication. + ### Development ```bash -python app.py +./run.sh # main app on 8000 +./run.sh main 8000 +./run.sh sensors 8001 ``` -The server starts on `http://localhost:5000`. - ### Production ```bash -# Install dependencies (includes gunicorn) pip install -r requirements.txt -# Run with gunicorn -gunicorn -c gunicorn.conf.py "app:create_app()" +./run-prod.sh main 8000 +./run-prod.sh sensors 8001 ``` -Gunicorn starts on `http://0.0.0.0:8000` with `cpu * 2 + 1` workers. +Both scripts use `.venv` when present. `run-prod.sh` runs the matching +migrations before starting each service. -#### systemd service +To expose the sensors API on the network: ```bash -sudo cp yggdrasil.service /etc/systemd/system/ -sudo systemctl daemon-reload -sudo systemctl enable --now yggdrasil +HOST=0.0.0.0 ./run.sh sensors 8001 +# or +BIND=0.0.0.0:8001 ./run-prod.sh sensors 8001 ``` -| Route | Description | -|---|---| -| `/sensors` | Dashboard with interactive chart and data table | -| `/api/docs` | API documentation with curl examples | +### Configuration -## Database +| Variable | Default | Purpose | +|---|---|---| +| `SENSORS_API_URL` | `http://localhost:8001` | Sensors API address used by the main app | +| `SENSORS_DATABASE_URL` | `sqlite:////sensors.db` | Sensors database URL | +| `SENSORS_DB_PATH` | `/sensors.db` | SQLite path when URL is unset | +| `USERS_DATABASE_URL` | `sqlite:////main_app.db` | Users database URL | +| `USERS_DB_PATH` | `/main_app.db` | SQLite path when URL is unset | +| `SECRET_KEY` | `.secret_key` file | Main app session signing key | +| `HOST` | main `0.0.0.0`, sensors `127.0.0.1` | Listen address | +| `BIND` | `:` | Gunicorn bind override | -Two tables are used: +`DATABASE_URL`/`DB_PATH` are accepted as legacy aliases for the sensors DB. + +When the services run on different machines, point `SENSORS_API_URL` at the +address reachable from the main app process. For example, with the sensors API +on your laptop (port 8001) and the main app on a VPS: + +```bash +# On the laptop: expose the local sensors API as VPS port 5432 +ssh -R 5432:localhost:8001 vps + +# On the VPS +SENSORS_API_URL=http://localhost:5432 ./run-prod.sh main 8000 +``` + +### systemd services + +```bash +sudo cp yggdrasil.service yggdrasil-sensors.service /etc/systemd/system/ +sudo systemctl daemon-reload +sudo systemctl enable --now yggdrasil yggdrasil-sensors +``` + +| Service | Port | Routes | +|---|---|---| +| main | 8000 | `/`, `/login`, `/home`, `/sensors`, `/api/docs`, `/api/*` (proxy) | +| sensors | 8001 | `/api/*` (sensors and readouts) | + +### Tests + +```bash +python -m unittest discover -s tests -v +``` + +## Databases + +### sensors.db + +Owned by the sensors app. Contains only sensor data. **sensor** | Column | Type | Description | @@ -137,64 +238,150 @@ Two tables are used: | `value` | Float | Measured value | | `timestamp` | DateTime | Readout timestamp (UTC) | +### main_app.db (users) + +Owned by the main app. Contains only accounts. + +**user** +| Column | Type | Description | +|---|---|---| +| `id` | Integer | Primary key | +| `username` | String(80) | Unique username | +| `password_hash` | String(255) | Password hash | +| `is_admin` | Boolean | Administrator flag | +| `created_at` | DateTime | Creation timestamp (UTC) | + +### Inspecting the databases on Linux + +The databases are ordinary SQLite files. Use the `sqlite3` CLI (install it with +`sudo apt install sqlite3` on Debian/Ubuntu): + +```bash +# List tables +sqlite3 sensors.db ".tables" +sqlite3 main_app.db ".tables" + +# Show the schema (all tables or one table) +sqlite3 sensors.db ".schema" +sqlite3 sensors.db ".schema sensor_readout" + +# Show columns of one table +sqlite3 sensors.db "PRAGMA table_info(sensor);" + +# Inspect rows (readable column output) +sqlite3 -header -column sensors.db \ + "SELECT id, name, created_at FROM sensor LIMIT 10;" +sqlite3 -header -column sensors.db \ + "SELECT id, sensor_id, type, value, timestamp FROM sensor_readout + ORDER BY timestamp DESC LIMIT 10;" + +# Users (do not print password_hash unnecessarily) +sqlite3 -header -column main_app.db \ + "SELECT id, username, is_admin, created_at FROM user;" + +# Counts and migration version +sqlite3 sensors.db "SELECT COUNT(*) FROM sensor_readout;" +sqlite3 sensors.db "SELECT * FROM alembic_version;" +sqlite3 main_app.db "SELECT * FROM alembic_version;" +``` + +Inside an interactive `sqlite3` session you can also use: + +```sql +.tables -- list tables +.headers on -- show column names +.mode column -- align columns +.schema sensor -- show CREATE TABLE +SELECT * FROM sensor; -- run a query +.quit -- exit +``` + +For read-only inspection add `-readonly`: + +```bash +sqlite3 -readonly -header -column sensors.db \ + "SELECT * FROM sensor_readout LIMIT 5;" +``` + +If `sqlite3` is not available, the same tables can be listed with Python: + +```bash +.venv/bin/python - <<'PY' +import sqlite3 +con = sqlite3.connect("sensors.db") +print([row[0] for row in con.execute( + "SELECT name FROM sqlite_master WHERE type='table'" +)]) +PY +``` + ### Migrations ```bash -# Create a new migration after model changes +# Sensors DB +alembic upgrade head alembic revision --autogenerate -m "description" -# Apply pending migrations -alembic upgrade head - -# Roll back one step -alembic downgrade -1 +# Users DB +alembic -c alembic_users.ini upgrade head +alembic -c alembic_users.ini revision --autogenerate -m "description" ``` ## API -Base URL: `http://localhost:5000/api` +Public API (main app): `http://localhost:8000/api` +Sensors API (local machine): `http://localhost:8001/api` + +The main app proxies every `/api/sensors*` request to the sensors API, so +clients and devices can keep using port 8000. ### Endpoints -| Method | Path | Description | -|---|---|---| -| `GET` | `/api/sensors` | List all sensors | -| `POST` | `/api/sensors` | Create a new sensor | -| `GET` | `/api/sensors/:id/types` | List readout types for a sensor | -| `GET` | `/api/sensors/:id/readouts` | Latest readouts (max 300, filterable by `?type=`) | -| `GET` | `/api/sensors/:id/readouts/hourly` | Hourly averaged readouts (filterable by `?type=`) | -| `POST` | `/api/sensors/:id/readouts` | Add a new readout | +| Method | Path | Auth | Description | +|---|---|---|---| +| `GET` | `/api/sensors` | public | List all sensors | +| `POST` | `/api/sensors` | public | Create a new sensor | +| `GET` | `/api/sensors/:id/types` | login | List readout types for a sensor | +| `GET` | `/api/sensors/:id/readouts` | login | Latest readouts (max 300, filterable by `?type=`) | +| `GET` | `/api/sensors/:id/readouts/hourly` | login | Hourly averaged readouts | +| `POST` | `/api/sensors/:id/readouts` | public | Add a new readout | + +Calling the sensors API directly on `:8001` skips login; it is open to every +local user by design. ### Examples ```bash # List sensors -curl http://localhost:5000/api/sensors +curl http://localhost:8000/api/sensors # Create a sensor -curl -X POST http://localhost:5000/api/sensors \ +curl -X POST http://localhost:8000/api/sensors \ -H "Content-Type: application/json" \ -d '{"name": "bmp280"}' -# Get temperature readouts for sensor 1 -curl "http://localhost:5000/api/sensors/1/readouts?type=temperature" +# Get temperature readouts for sensor 1 (requires a logged-in session) +curl -b cookies.txt "http://localhost:8000/api/sensors/1/readouts?type=temperature" -# Get hourly averaged data -curl "http://localhost:5000/api/sensors/1/readouts/hourly?type=temperature" +# Get hourly averaged data (requires a logged-in session) +curl -b cookies.txt "http://localhost:8000/api/sensors/1/readouts/hourly?type=temperature" # Add a readout -curl -X POST http://localhost:5000/api/sensors/1/readouts \ +curl -X POST http://localhost:8000/api/sensors/1/readouts \ -H "Content-Type: application/json" \ -d '{"type": "temperature", "value": 22.5}' + +# Direct local access with no login +curl http://localhost:8001/api/sensors/1/readouts ``` Full API documentation with request/response examples is available at `/api/docs`. ## Frontend -The dashboard at `/sensors` provides: +The dashboard in the main app at `/sensors` provides: - **Sensor dropdown** — select a sensor from the list - **Type dropdown** — select a readout type (populated based on the chosen sensor) -- **Line chart** — hourly averaged values rendered with Chart.js +- **Line chart** — raw or hourly-averaged values for the selected range - **Data table** — raw readouts with formatted timestamps (latest 300 entries) diff --git a/server/alembic/env.py b/server/alembic/env.py index dbb7b34..c221c2f 100644 --- a/server/alembic/env.py +++ b/server/alembic/env.py @@ -8,7 +8,7 @@ from sqlalchemy import engine_from_config, pool # Add the project root to sys.path so that models can be imported. sys.path.insert(0, os.path.realpath(os.path.join(os.path.dirname(__file__), ".."))) -from app import create_app +from sensors_app import create_app from models import db # Create the Flask application to get the database URI. diff --git a/server/alembic/versions/c59e689377c2_add_user_table.py b/server/alembic/versions/c59e689377c2_add_user_table.py index db94792..896f74f 100644 --- a/server/alembic/versions/c59e689377c2_add_user_table.py +++ b/server/alembic/versions/c59e689377c2_add_user_table.py @@ -1,4 +1,4 @@ -"""add user table +"""deprecated user table (users moved to the main app) Revision ID: c59e689377c2 Revises: 7f92bfc47dfc @@ -7,9 +7,6 @@ Create Date: 2026-10-03 19:06:44.095699 """ from typing import Sequence, Union -from alembic import op -import sqlalchemy as sa - # revision identifiers, used by Alembic. revision: str = 'c59e689377c2' @@ -19,22 +16,12 @@ depends_on: Union[str, Sequence[str], None] = None def upgrade() -> None: - """Upgrade schema.""" - # ### commands auto generated by Alembic - please adjust! ### - op.create_table('user', - sa.Column('id', sa.Integer(), nullable=False), - sa.Column('username', sa.String(length=80), nullable=False), - sa.Column('password_hash', sa.String(length=255), nullable=False), - sa.Column('is_admin', sa.Boolean(), nullable=False), - sa.Column('created_at', sa.DateTime(), nullable=True), - sa.PrimaryKeyConstraint('id'), - sa.UniqueConstraint('username') - ) - # ### end Alembic commands ### + """No-op: the sensors database no longer stores users. + + The revision is kept so legacy databases that already point at it can + still be upgraded. New sensors databases create only sensor tables. + """ def downgrade() -> None: - """Downgrade schema.""" - # ### commands auto generated by Alembic - please adjust! ### - op.drop_table('user') - # ### end Alembic commands ### + """No-op: the user table is no longer part of the sensors database.""" diff --git a/server/alembic_users.ini b/server/alembic_users.ini new file mode 100644 index 0000000..6d0d3fd --- /dev/null +++ b/server/alembic_users.ini @@ -0,0 +1,37 @@ +[alembic] +script_location = %(here)s/alembic_users +prepend_sys_path = . + +[loggers] +keys = root,sqlalchemy,alembic + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = WARN +handlers = console +qualname = + +[logger_sqlalchemy] +level = WARN +handlers = +qualname = sqlalchemy.engine + +[logger_alembic] +level = INFO +handlers = +qualname = alembic + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(levelname)-5.5s [%(name)s] %(message)s +datefmt = %H:%M:%S diff --git a/server/alembic_users/env.py b/server/alembic_users/env.py new file mode 100644 index 0000000..faafb2f --- /dev/null +++ b/server/alembic_users/env.py @@ -0,0 +1,52 @@ +import os +import sys +from logging.config import fileConfig + +from alembic import context +from sqlalchemy import engine_from_config, pool + +sys.path.insert(0, os.path.realpath(os.path.join(os.path.dirname(__file__), ".."))) + +from config import users_database_uri +from user_models import user_db + +config = context.config +config.set_main_option("sqlalchemy.url", users_database_uri()) + +if config.config_file_name is not None: + fileConfig(config.config_file_name) + +target_metadata = user_db.metadata + + +def run_migrations_offline() -> None: + url = config.get_main_option("sqlalchemy.url") + context.configure( + url=url, + target_metadata=target_metadata, + literal_binds=True, + dialect_opts={"paramstyle": "named"}, + ) + + with context.begin_transaction(): + context.run_migrations() + + +def run_migrations_online() -> None: + connectable = engine_from_config( + config.get_section(config.config_ini_section, {}), + prefix="sqlalchemy.", + poolclass=pool.NullPool, + ) + + with connectable.connect() as connection: + context.configure(connection=connection, target_metadata=target_metadata) + + with context.begin_transaction(): + context.run_migrations() + + +if context.is_offline_mode(): + run_migrations_offline() +else: + run_migrations_online() diff --git a/server/alembic_users/script.py.mako b/server/alembic_users/script.py.mako new file mode 100644 index 0000000..1101630 --- /dev/null +++ b/server/alembic_users/script.py.mako @@ -0,0 +1,28 @@ +"""${message} + +Revision ID: ${up_revision} +Revises: ${down_revision | comma,n} +Create Date: ${create_date} + +""" +from typing import Sequence, Union + +from alembic import op +import sqlalchemy as sa +${imports if imports else ""} + +# revision identifiers, used by Alembic. +revision: str = ${repr(up_revision)} +down_revision: Union[str, Sequence[str], None] = ${repr(down_revision)} +branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)} +depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)} + + +def upgrade() -> None: + """Upgrade schema.""" + ${upgrades if upgrades else "pass"} + + +def downgrade() -> None: + """Downgrade schema.""" + ${downgrades if downgrades else "pass"} diff --git a/server/alembic_users/versions/1a7c3e5d9b21_create_user_table.py b/server/alembic_users/versions/1a7c3e5d9b21_create_user_table.py new file mode 100644 index 0000000..a994e5b --- /dev/null +++ b/server/alembic_users/versions/1a7c3e5d9b21_create_user_table.py @@ -0,0 +1,35 @@ +"""create user table + +Revision ID: 1a7c3e5d9b21 +Revises: +Create Date: 2026-11-01 12:00:00.000000 + +""" +from typing import Sequence, Union + +from alembic import op +import sqlalchemy as sa + + +# revision identifiers, used by Alembic. +revision: str = "1a7c3e5d9b21" +down_revision: Union[str, Sequence[str], None] = None +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + op.create_table( + "user", + sa.Column("id", sa.Integer(), nullable=False), + sa.Column("username", sa.String(length=80), nullable=False), + sa.Column("password_hash", sa.String(length=255), nullable=False), + sa.Column("is_admin", sa.Boolean(), nullable=False), + sa.Column("created_at", sa.DateTime(), nullable=True), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("username"), + ) + + +def downgrade() -> None: + op.drop_table("user") diff --git a/server/app.py b/server/app.py index 9f93e9c..df1eb5d 100644 --- a/server/app.py +++ b/server/app.py @@ -1,52 +1,61 @@ import os -from datetime import datetime, timedelta, timezone -from zoneinfo import ZoneInfo, ZoneInfoNotFoundError import click -from flask import Flask, jsonify, redirect, render_template, request, url_for -from flask_login import ( - LoginManager, - current_user, - login_required, - login_user, - logout_user, +from flask import ( + Flask, + jsonify, + redirect, + render_template, + request, + url_for, ) +from flask_login import current_user, login_required, login_user, logout_user -from models import Sensor, SensorReadout, User, db - -login_manager = LoginManager() +from auth import init_login_manager +from config import load_secret_key, sensors_api_url, users_database_uri +from sensors_client import SensorsAPIError, SensorsClient +from user_models import User, user_db def create_app(): app = Flask(__name__) - db_path = os.path.join(os.path.dirname(__file__), "data.db") - app.config["SQLALCHEMY_DATABASE_URI"] = f"sqlite:///{db_path}" + app.config["SECRET_KEY"] = load_secret_key() + app.config["SQLALCHEMY_DATABASE_URI"] = users_database_uri() app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False - app.config["SECRET_KEY"] = os.environ.get("SECRET_KEY", "dev-secret-key-change-me") + app.config["SENSORS_API_URL"] = sensors_api_url() - db.init_app(app) - login_manager.init_app(app) - login_manager.login_view = "login" + user_db.init_app(app) + sensors = SensorsClient(app.config["SENSORS_API_URL"]) - @login_manager.user_loader - def load_user(user_id): - return db.session.get(User, int(user_id)) - - @login_manager.unauthorized_handler - def unauthorized(): - # Return JSON for API calls instead of redirecting to the login page. - if request.path.startswith("/api/"): - return jsonify({"error": "authentication required"}), 401 - next_page = request.path + def _login_redirect(req): login_url = url_for("login") - if next_page and next_page != login_url: - login_url += f"?next={next_page}" - return redirect(login_url) + if req.path and req.path != login_url: + return url_for("login", next=req.path) + return login_url + + def _load_user(user_id): + try: + return user_db.session.get(User, int(user_id)) + except (TypeError, ValueError): + return None + + init_login_manager(app, _login_redirect, _load_user) def _is_safe_next(target: str | None) -> bool: return bool(target) and target.startswith("/") and not target.startswith("//") + def _proxy(call): + try: + status, data = call() + except SensorsAPIError as exc: + return jsonify({"error": str(exc)}), 502 + return jsonify(data), status + + @app.context_processor + def inject_service_urls(): + return {"main_api_url": request.url_root.rstrip("/") + "/api"} + # --- Pages --- @app.route("/") @@ -94,176 +103,49 @@ def create_app(): def api_docs(): return render_template("docs.html", user=current_user) - # --- API: device endpoints (intentionally public) --- - # The sensor firmware has no browser session, but it needs the sensor - # list to resolve a name to an id before pushing readouts. + # --- API: proxied to the sensors service --- + # The browser only talks to this app; the sensors service owns the + # sensor database and the actual API implementation. @app.route("/api/sensors") def api_sensors(): - sensors = Sensor.query.order_by(Sensor.name).all() - return jsonify([{"id": s.id, "name": s.name} for s in sensors]) + return _proxy(sensors.list_sensors) - # --- API: dashboard reads (login required, the browser session cookie is enough) --- + @app.route("/api/sensors", methods=["POST"]) + def api_create_sensor(): + return _proxy(lambda: sensors.create_sensor(request.get_json(silent=True))) + + @app.route("/api/sensors//readouts", methods=["POST"]) + def api_create_readout(sensor_id): + return _proxy( + lambda: sensors.create_readout(sensor_id, request.get_json(silent=True)) + ) @app.route("/api/sensors//types") @login_required def api_sensor_types(sensor_id): - types = ( - db.session.query(SensorReadout.type) - .filter_by(sensor_id=sensor_id) - .distinct() - .order_by(SensorReadout.type) - .all() - ) - return jsonify([t[0] for t in types]) + return _proxy(lambda: sensors.sensor_types(sensor_id)) @app.route("/api/sensors//readouts") @login_required def api_sensor_readouts(sensor_id): - type_ = request.args.get("type") - query = SensorReadout.query.filter_by(sensor_id=sensor_id) - if type_: - query = query.filter_by(type=type_) - readouts = query.order_by(SensorReadout.timestamp.desc()).limit(300).all() - return jsonify( - [ - {"value": r.value, "timestamp": r.timestamp.isoformat() + "Z"} - for r in readouts - ] + return _proxy( + lambda: sensors.sensor_readouts( + sensor_id, {"type": request.args.get("type")} + ) ) @app.route("/api/sensors//readouts/hourly") @login_required def api_sensor_readouts_hourly(sensor_id): - type_ = request.args.get("type") - range_ = request.args.get("range", "all") - # Frontend wysyła strefę czasową przeglądarki (np. Europe/Warsaw), - # żeby "today" znaczyło "od północy czasu lokalnego", a nie UTC. - tz_name = request.args.get("tz", "Europe/Warsaw") or "Europe/Warsaw" - try: - local_tz = ZoneInfo(tz_name) - except (ZoneInfoNotFoundError, ValueError): - return jsonify({"error": f"unknown timezone: {tz_name}"}), 400 - # Timestamps are stored as naive UTC, so the cutoff must be naive UTC too. - now_utc = datetime.now(timezone.utc) - now_local = now_utc.astimezone(local_tz) - if range_ == "today": - midnight_local = now_local.replace(hour=0, minute=0, second=0, microsecond=0) - cutoff = midnight_local.astimezone(timezone.utc).replace(tzinfo=None) - elif range_ == "24h": - cutoff = (now_utc - timedelta(hours=24)).replace(tzinfo=None) - elif range_ == "week": - cutoff = (now_utc - timedelta(days=7)).replace(tzinfo=None) - elif range_ == "month": - cutoff = (now_utc - timedelta(days=30)).replace(tzinfo=None) - elif range_ == "all": - cutoff = None - else: - return jsonify({"error": "range must be one of: today, 24h, week, month, all"}), 400 - query = db.session.query(SensorReadout.timestamp, SensorReadout.value).filter( - SensorReadout.sensor_id == sensor_id + return _proxy( + lambda: sensors.sensor_readouts_hourly(sensor_id, request.args.to_dict()) ) - if cutoff is not None: - query = query.filter(SensorReadout.timestamp >= cutoff) - if type_: - query = query.filter(SensorReadout.type == type_) - rows = query.order_by(SensorReadout.timestamp).all() - # Zakresy today i 24h domyślnie zwracają wszystkie pomiary bez - # uśredniania; checkbox "hourly average" wymusza średnie godzinowe. - averaged = range_ not in ("today", "24h") or request.args.get( - "avg", "0" - ).lower() in ("1", "true", "yes") - if not averaged: - return jsonify( - [ - { - "timestamp": ts.strftime("%Y-%m-%dT%H:%M:%S"), - "value": value, - } - for ts, value in rows - ] - ) - # Pozostałe zakresy: średnia godzinowa liczona po pełnej godzinie - # lokalnej (poprawne też przy zmianie DST), znacznik to początek - # godziny w UTC (frontend dokleja "Z"). - buckets: dict[datetime, list[float]] = {} - for ts, value in rows: - ts_utc = ts.replace(tzinfo=timezone.utc) - hour_local = ts_utc.astimezone(local_tz).replace( - minute=0, second=0, microsecond=0 - ) - hour_utc = hour_local.astimezone(timezone.utc).replace(tzinfo=None) - buckets.setdefault(hour_utc, []).append(value) - result = [ - { - "timestamp": hour.strftime("%Y-%m-%dT%H:00:00"), - "value": round(sum(values) / len(values), 2), - } - for hour, values in sorted(buckets.items()) - ] - return jsonify(result) - - # --- API: device writes (intentionally public — called by devices/sensors) --- - - @app.route("/api/sensors", methods=["POST"]) - def api_create_sensor(): - data = request.get_json(silent=True) - if not data or "name" not in data: - return jsonify({"error": "name is required"}), 400 - - name = data["name"].strip() - if not name: - return jsonify({"error": "name cannot be empty"}), 400 - - if Sensor.query.filter_by(name=name).first(): - return jsonify({"error": "sensor already exists"}), 409 - - sensor = Sensor(name=name) - db.session.add(sensor) - db.session.commit() - return jsonify({"id": sensor.id, "name": sensor.name}), 201 - - @app.route("/api/sensors//readouts", methods=["POST"]) - def api_create_readout(sensor_id): - sensor = db.session.get(Sensor, sensor_id) - if not sensor: - return jsonify({"error": "sensor not found"}), 404 - - data = request.get_json(silent=True) - if not data: - return jsonify({"error": "request body is required"}), 400 - - missing = [f for f in ("type", "value") if f not in data] - if missing: - return jsonify({"error": f"missing fields: {', '.join(missing)}"}), 400 - - try: - value = float(data["value"]) - except (TypeError, ValueError): - return jsonify({"error": "value must be a number"}), 400 - - readout = SensorReadout( - sensor_id=sensor_id, - type=str(data["type"]).strip(), - value=value, - ) - db.session.add(readout) - db.session.commit() - return jsonify( - { - "id": readout.id, - "sensor_id": readout.sensor_id, - "type": readout.type, - "value": readout.value, - "timestamp": readout.timestamp.isoformat() + "Z", - } - ), 201 # --- Command for creating the first admin --- # Usage: # flask --app app create-admin --username admin --password 'secret' # ADMIN_USERNAME=admin ADMIN_PASSWORD=secret flask --app app create-admin - # flask --app app create-admin (asks interactively) @app.cli.command("create-admin") @click.option("--username", default=None, help="Administrator username") @@ -287,13 +169,13 @@ def create_app(): if user: user.set_password(password) user.is_admin = True - db.session.commit() + user_db.session.commit() click.echo(f"Updated existing user '{username}' (granted is_admin=True).") else: user = User(username=username, is_admin=True) user.set_password(password) - db.session.add(user) - db.session.commit() + user_db.session.add(user) + user_db.session.commit() click.echo(f"Created administrator '{username}'.") return app @@ -301,4 +183,4 @@ def create_app(): if __name__ == "__main__": app = create_app() - app.run(debug=True, host='0.0.0.0') + app.run(debug=True, host='0.0.0.0', port=8000) diff --git a/server/auth.py b/server/auth.py new file mode 100644 index 0000000..b265465 --- /dev/null +++ b/server/auth.py @@ -0,0 +1,17 @@ +from flask import jsonify, redirect, request +from flask_login import LoginManager + + +def init_login_manager(app, login_url_builder, load_user, api_prefixes=("/api/",)): + login_manager = LoginManager() + login_manager.init_app(app) + login_manager.user_loader(load_user) + + @login_manager.unauthorized_handler + def unauthorized(): + # API clients get JSON instead of a redirect to the login page. + if request.path.startswith(tuple(api_prefixes)): + return jsonify({"error": "authentication required"}), 401 + return redirect(login_url_builder(request)) + + return login_manager diff --git a/server/config.py b/server/config.py new file mode 100644 index 0000000..5758635 --- /dev/null +++ b/server/config.py @@ -0,0 +1,45 @@ +import os +from pathlib import Path + +BASE_DIR = Path(__file__).resolve().parent +SECRET_FILE = BASE_DIR / ".secret_key" + +DEFAULT_SENSORS_PORT = 8001 + + +def load_secret_key() -> str: + """Return a stable key used to sign the main app session cookies.""" + key = os.environ.get("SECRET_KEY") + if key: + return key + if SECRET_FILE.exists(): + return SECRET_FILE.read_text().strip() + return "dev-secret-key-change-me" + + +def sensors_database_uri() -> str: + """Database owned by the sensors API (sensors and readouts only).""" + configured = os.environ.get("SENSORS_DATABASE_URL") or os.environ.get("DATABASE_URL") + if configured: + return configured + db_path = os.environ.get( + "SENSORS_DB_PATH", os.environ.get("DB_PATH", str(BASE_DIR / "sensors.db")) + ) + return f"sqlite:///{db_path}" + + +def users_database_uri() -> str: + """Database owned by the main app (user accounts only).""" + configured = os.environ.get("USERS_DATABASE_URL") + if configured: + return configured + db_path = os.environ.get("USERS_DB_PATH", str(BASE_DIR / "main_app.db")) + return f"sqlite:///{db_path}" + + +def sensors_api_url() -> str: + """Base URL used by the main app to call the sensors API (server-to-server).""" + configured = os.environ.get("SENSORS_API_URL") + if configured: + return configured.rstrip("/") + return f"http://localhost:{DEFAULT_SENSORS_PORT}" diff --git a/server/gunicorn.conf.py b/server/gunicorn.conf.py index a8365eb..2bf976e 100644 --- a/server/gunicorn.conf.py +++ b/server/gunicorn.conf.py @@ -1,6 +1,6 @@ import os -bind = "0.0.0.0:8000" +bind = os.environ.get("BIND") or f"0.0.0.0:{os.environ.get('PORT', '8000')}" workers = min(os.cpu_count() * 2 + 1, 9) timeout = 30 keepalive = 5 diff --git a/server/migrate_legacy_db.py b/server/migrate_legacy_db.py new file mode 100644 index 0000000..c08cfeb --- /dev/null +++ b/server/migrate_legacy_db.py @@ -0,0 +1,73 @@ +"""Copy data from a legacy combined data.db into the split databases. + +The legacy database contained users, sensors and readouts in one file. Run the +new migrations first, then import the old rows: + + alembic upgrade head + alembic -c alembic_users.ini upgrade head + python migrate_legacy_db.py + +The legacy file is only read; nothing is deleted from it. +""" + +import argparse +from pathlib import Path + +from sqlalchemy import MetaData, Table, create_engine, inspect, select +from sqlalchemy.dialects.sqlite import insert + +from models import Sensor, SensorReadout +from user_models import User + + +def _sqlite_engine(path: Path): + if not path.exists(): + raise SystemExit(f"database not found: {path}") + return create_engine(f"sqlite:///{path}") + + +def _copy_table(source, destination, table) -> int: + if table.name not in inspect(source).get_table_names(): + return 0 + + source_table = Table(table.name, MetaData(), autoload_with=source) + with source.connect() as conn: + rows = [dict(row._mapping) for row in conn.execute(select(source_table))] + if not rows: + return 0 + + with destination.begin() as conn: + conn.execute(insert(table).on_conflict_do_nothing(), rows) + return len(rows) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--legacy", default="data.db", type=Path) + parser.add_argument("--sensors", default="sensors.db", type=Path) + parser.add_argument("--main-app", default="main_app.db", type=Path) + args = parser.parse_args() + + legacy = _sqlite_engine(args.legacy) + sensors = _sqlite_engine(args.sensors) + main_app = _sqlite_engine(args.main_app) + + if not inspect(sensors).has_table(Sensor.__tablename__): + raise SystemExit(f"{args.sensors} has no schema; run alembic upgrade head first") + if not inspect(main_app).has_table(User.__tablename__): + raise SystemExit( + f"{args.main_app} has no schema; run alembic -c alembic_users.ini upgrade head first" + ) + + sensor_count = _copy_table(legacy, sensors, Sensor.__table__) + readout_count = _copy_table(legacy, sensors, SensorReadout.__table__) + user_count = _copy_table(legacy, main_app, User.__table__) + + print( + f"Copied {sensor_count} sensors, {readout_count} readouts and " + f"{user_count} users. The legacy database was not modified." + ) + + +if __name__ == "__main__": + main() diff --git a/server/models.py b/server/models.py index 547dd91..f54cb28 100644 --- a/server/models.py +++ b/server/models.py @@ -1,28 +1,10 @@ from datetime import datetime, timezone -from flask_login import UserMixin from flask_sqlalchemy import SQLAlchemy -from werkzeug.security import check_password_hash, generate_password_hash db = SQLAlchemy() -class User(UserMixin, db.Model): - __tablename__ = "user" - - id = db.Column(db.Integer, primary_key=True) - username = db.Column(db.String(80), nullable=False, unique=True) - password_hash = db.Column(db.String(255), nullable=False) - is_admin = db.Column(db.Boolean, nullable=False, default=False) - created_at = db.Column(db.DateTime, default=lambda: datetime.now(timezone.utc)) - - def set_password(self, password: str) -> None: - self.password_hash = generate_password_hash(password) - - def check_password(self, password: str) -> bool: - return check_password_hash(self.password_hash, password) - - class Sensor(db.Model): __tablename__ = "sensor" diff --git a/server/run-prod.sh b/server/run-prod.sh index 4c13c2f..8dd99ed 100755 --- a/server/run-prod.sh +++ b/server/run-prod.sh @@ -1,35 +1,68 @@ #!/usr/bin/env bash -# Production startup (gunicorn) with a stable SECRET_KEY. +# Production startup (gunicorn). Pick the service and port: # -# Key resolution order: +# ./run-prod.sh # main app on 8000 +# ./run-prod.sh main 8000 # main app on 8000 +# ./run-prod.sh sensors 8001 # sensors app on 8001 (localhost only) +# +# SECRET_KEY resolution (main app sessions): # 1. SECRET_KEY environment variable (takes precedence), # 2. .secret_key file next to this script (created once on first start), # 3. otherwise a random key is generated and saved to .secret_key. # -# A stable key matters: it signs the session cookies, so changing it -# on every restart would log out all users. +# The sensors API is open to every user on the local machine. To expose it +# on the network explicitly set HOST=0.0.0.0 (or BIND). set -euo pipefail cd "$(dirname "$0")" -SECRET_FILE=".secret_key" +APP="${1:-${APP:-main}}" +PORT="${2:-${PORT:-}}" -if [ -z "${SECRET_KEY:-}" ]; then - if [ -f "$SECRET_FILE" ]; then - SECRET_KEY="$(cat "$SECRET_FILE")" - else - SECRET_KEY="$(python3 -c 'import secrets; print(secrets.token_hex(32))')" - printf '%s' "$SECRET_KEY" > "$SECRET_FILE" - chmod 600 "$SECRET_FILE" - echo "Generated a new SECRET_KEY, saved to $SECRET_FILE" >&2 +case "$APP" in + main) + MODULE="app:create_app" + DEFAULT_PORT=8000 + DEFAULT_HOST="0.0.0.0" + ;; + sensors) + MODULE="sensors_app:create_app" + DEFAULT_PORT=8001 + DEFAULT_HOST="127.0.0.1" + ;; + *) + echo "Usage: $0 [main|sensors] [port]" >&2 + exit 2 + ;; +esac + +PORT="${PORT:-$DEFAULT_PORT}" +HOST="${HOST:-$DEFAULT_HOST}" + +if [ "$APP" = "main" ]; then + SECRET_FILE=".secret_key" + if [ -z "${SECRET_KEY:-}" ]; then + if [ -f "$SECRET_FILE" ]; then + SECRET_KEY="$(cat "$SECRET_FILE")" + else + SECRET_KEY="$(python3 -c 'import secrets; print(secrets.token_hex(32))')" + printf '%s' "$SECRET_KEY" > "$SECRET_FILE" + chmod 600 "$SECRET_FILE" + echo "Generated a new SECRET_KEY, saved to $SECRET_FILE" >&2 + fi + export SECRET_KEY fi - export SECRET_KEY fi -# Migrate before starting, so the schema matches the code. +# Each service runs migrations for its own database. if [ -x ".venv/bin/alembic" ]; then - .venv/bin/alembic upgrade head -elif command -v alembic >/dev/null 2>&1; then - alembic upgrade head + ALEMBIC=".venv/bin/alembic" +else + ALEMBIC="alembic" +fi +if [ "$APP" = "main" ]; then + "$ALEMBIC" -c alembic_users.ini upgrade head +else + "$ALEMBIC" -c alembic.ini upgrade head fi if [ -x ".venv/bin/gunicorn" ]; then @@ -38,4 +71,8 @@ else GUNICORN="gunicorn" fi -exec "$GUNICORN" -c gunicorn.conf.py "app:create_app()" +export BIND="${BIND:-$HOST:$PORT}" +echo "Starting $APP app on http://$BIND" >&2 +# Gunicorn needs the factory call syntax ("module:create_app()"), otherwise it +# treats create_app itself as the WSGI callable. +exec "$GUNICORN" -c gunicorn.conf.py "$MODULE()" diff --git a/server/run.sh b/server/run.sh new file mode 100755 index 0000000..3a2c0b4 --- /dev/null +++ b/server/run.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash +# Development launcher. Pick the service and port: +# +# ./run.sh # main app on 8000 +# ./run.sh main 8000 # main app on 8000 +# ./run.sh sensors 8001 # sensors app on 8001 (localhost only) +# +# The main app calls the sensors API server-to-server. Override its address +# when the sensors app is not on localhost:8001: +# +# SENSORS_API_URL=http://192.168.0.50:9001 ./run.sh main 8000 +# +# The sensors API is open to every user on the local machine. To expose it +# on the network explicitly set HOST=0.0.0.0. +set -euo pipefail +cd "$(dirname "$0")" + +APP="${1:-${APP:-main}}" +PORT="${2:-${PORT:-}}" + +case "$APP" in + main) + MODULE="app:create_app" + DEFAULT_PORT=8000 + DEFAULT_HOST="0.0.0.0" + ;; + sensors) + MODULE="sensors_app:create_app" + DEFAULT_PORT=8001 + DEFAULT_HOST="127.0.0.1" + ;; + *) + echo "Usage: $0 [main|sensors] [port]" >&2 + exit 2 + ;; +esac + +PORT="${PORT:-$DEFAULT_PORT}" +HOST="${HOST:-$DEFAULT_HOST}" + +if [ -x ".venv/bin/python" ]; then + PYTHON=".venv/bin/python" +else + PYTHON="python3" +fi + +echo "Starting $APP app on http://$HOST:$PORT" >&2 +exec "$PYTHON" -m flask --app "$MODULE" run --host "$HOST" --port "$PORT" --debug diff --git a/server/seed.py b/server/seed.py index cecd693..36473f3 100644 --- a/server/seed.py +++ b/server/seed.py @@ -3,7 +3,7 @@ import random from datetime import datetime, timedelta, timezone -from app import create_app +from sensors_app import create_app from models import Sensor, SensorReadout, db diff --git a/server/sensors_app.py b/server/sensors_app.py new file mode 100644 index 0000000..10e692f --- /dev/null +++ b/server/sensors_app.py @@ -0,0 +1,183 @@ +from datetime import datetime, timedelta, timezone +from zoneinfo import ZoneInfo, ZoneInfoNotFoundError + +from flask import Flask, jsonify, request + +from config import sensors_database_uri +from models import Sensor, SensorReadout, db + + +def create_app(): + app = Flask(__name__) + + app.config["SQLALCHEMY_DATABASE_URI"] = sensors_database_uri() + app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False + + db.init_app(app) + + # --- API: device endpoints (intentionally public) --- + # The sensor firmware has no browser session, but it needs the sensor + # list to resolve a name to an id before pushing readouts. + + @app.route("/api/sensors") + def api_sensors(): + sensors = Sensor.query.order_by(Sensor.name).all() + return jsonify([{"id": s.id, "name": s.name} for s in sensors]) + + @app.route("/api/sensors", methods=["POST"]) + def api_create_sensor(): + data = request.get_json(silent=True) + if not data or "name" not in data: + return jsonify({"error": "name is required"}), 400 + + name = data["name"].strip() + if not name: + return jsonify({"error": "name cannot be empty"}), 400 + + if Sensor.query.filter_by(name=name).first(): + return jsonify({"error": "sensor already exists"}), 409 + + sensor = Sensor(name=name) + db.session.add(sensor) + db.session.commit() + return jsonify({"id": sensor.id, "name": sensor.name}), 201 + + @app.route("/api/sensors//readouts", methods=["POST"]) + def api_create_readout(sensor_id): + sensor = db.session.get(Sensor, sensor_id) + if not sensor: + return jsonify({"error": "sensor not found"}), 404 + + data = request.get_json(silent=True) + if not data: + return jsonify({"error": "request body is required"}), 400 + + missing = [f for f in ("type", "value") if f not in data] + if missing: + return jsonify({"error": f"missing fields: {', '.join(missing)}"}), 400 + + try: + value = float(data["value"]) + except (TypeError, ValueError): + return jsonify({"error": "value must be a number"}), 400 + + readout = SensorReadout( + sensor_id=sensor_id, + type=str(data["type"]).strip(), + value=value, + ) + db.session.add(readout) + db.session.commit() + return jsonify( + { + "id": readout.id, + "sensor_id": readout.sensor_id, + "type": readout.type, + "value": readout.value, + "timestamp": readout.timestamp.isoformat() + "Z", + } + ), 201 + + # --- API: dashboard reads (public on the local machine) --- + + @app.route("/api/sensors//types") + def api_sensor_types(sensor_id): + types = ( + db.session.query(SensorReadout.type) + .filter_by(sensor_id=sensor_id) + .distinct() + .order_by(SensorReadout.type) + .all() + ) + return jsonify([t[0] for t in types]) + + @app.route("/api/sensors//readouts") + def api_sensor_readouts(sensor_id): + type_ = request.args.get("type") + query = SensorReadout.query.filter_by(sensor_id=sensor_id) + if type_: + query = query.filter_by(type=type_) + readouts = query.order_by(SensorReadout.timestamp.desc()).limit(300).all() + return jsonify( + [ + {"value": r.value, "timestamp": r.timestamp.isoformat() + "Z"} + for r in readouts + ] + ) + + @app.route("/api/sensors//readouts/hourly") + def api_sensor_readouts_hourly(sensor_id): + type_ = request.args.get("type") + range_ = request.args.get("range", "all") + # The frontend sends the browser timezone (e.g. Europe/Warsaw), so + # "today" means "since midnight local time", not UTC. + tz_name = request.args.get("tz", "Europe/Warsaw") or "Europe/Warsaw" + try: + local_tz = ZoneInfo(tz_name) + except (ZoneInfoNotFoundError, ValueError): + return jsonify({"error": f"unknown timezone: {tz_name}"}), 400 + # Timestamps are stored as naive UTC, so the cutoff must be naive UTC too. + now_utc = datetime.now(timezone.utc) + now_local = now_utc.astimezone(local_tz) + if range_ == "today": + midnight_local = now_local.replace(hour=0, minute=0, second=0, microsecond=0) + cutoff = midnight_local.astimezone(timezone.utc).replace(tzinfo=None) + elif range_ == "24h": + cutoff = (now_utc - timedelta(hours=24)).replace(tzinfo=None) + elif range_ == "week": + cutoff = (now_utc - timedelta(days=7)).replace(tzinfo=None) + elif range_ == "month": + cutoff = (now_utc - timedelta(days=30)).replace(tzinfo=None) + elif range_ == "all": + cutoff = None + else: + return jsonify({"error": "range must be one of: today, 24h, week, month, all"}), 400 + query = db.session.query(SensorReadout.timestamp, SensorReadout.value).filter( + SensorReadout.sensor_id == sensor_id + ) + if cutoff is not None: + query = query.filter(SensorReadout.timestamp >= cutoff) + if type_: + query = query.filter(SensorReadout.type == type_) + rows = query.order_by(SensorReadout.timestamp).all() + # today/24h return every measurement by default; the "hourly average" + # checkbox forces hourly averages. + averaged = range_ not in ("today", "24h") or request.args.get( + "avg", "0" + ).lower() in ("1", "true", "yes") + if not averaged: + return jsonify( + [ + { + "timestamp": ts.strftime("%Y-%m-%dT%H:%M:%S"), + "value": value, + } + for ts, value in rows + ] + ) + # Other ranges: hourly average over the full local hour (correct + # across DST changes), timestamp is the start of the hour in UTC + # (the frontend appends "Z"). + buckets: dict[datetime, list[float]] = {} + for ts, value in rows: + ts_utc = ts.replace(tzinfo=timezone.utc) + hour_local = ts_utc.astimezone(local_tz).replace( + minute=0, second=0, microsecond=0 + ) + hour_utc = hour_local.astimezone(timezone.utc).replace(tzinfo=None) + buckets.setdefault(hour_utc, []).append(value) + result = [ + { + "timestamp": hour.strftime("%Y-%m-%dT%H:00:00"), + "value": round(sum(values) / len(values), 2), + } + for hour, values in sorted(buckets.items()) + ] + return jsonify(result) + + return app + + +if __name__ == "__main__": + app = create_app() + app.run(debug=True, host='127.0.0.1', port=8001) diff --git a/server/sensors_client.py b/server/sensors_client.py new file mode 100644 index 0000000..17d5517 --- /dev/null +++ b/server/sensors_client.py @@ -0,0 +1,68 @@ +import json +from urllib import error, parse, request + + +class SensorsAPIError(Exception): + """Raised when the sensors API is unreachable or returns invalid data.""" + + +class SensorsClient: + def __init__(self, base_url: str, timeout: float = 10.0): + self.base_url = base_url.rstrip("/") + self.timeout = timeout + + def _request(self, method, path, params=None, payload=None): + url = f"{self.base_url}{path}" + if params: + query = parse.urlencode({k: v for k, v in params.items() if v is not None}) + if query: + url += f"?{query}" + + data = None + headers = {"Accept": "application/json"} + if payload is not None: + data = json.dumps(payload).encode("utf-8") + headers["Content-Type"] = "application/json" + + req = request.Request(url, data=data, headers=headers, method=method) + try: + with request.urlopen(req, timeout=self.timeout) as response: + body = response.read() + return response.status, self._decode(body) + except error.HTTPError as exc: + return exc.code, self._decode(exc.read()) + except (error.URLError, TimeoutError, OSError) as exc: + raise SensorsAPIError(f"sensors API unavailable: {exc}") from exc + + @staticmethod + def _decode(body: bytes): + if not body: + return None + try: + return json.loads(body) + except json.JSONDecodeError as exc: + raise SensorsAPIError("sensors API returned invalid JSON") from exc + + # --- Sensor data --- + + def list_sensors(self): + return self._request("GET", "/api/sensors") + + def create_sensor(self, payload): + return self._request("POST", "/api/sensors", payload=payload) + + def create_readout(self, sensor_id, payload): + return self._request( + "POST", f"/api/sensors/{sensor_id}/readouts", payload=payload + ) + + def sensor_types(self, sensor_id): + return self._request("GET", f"/api/sensors/{sensor_id}/types") + + def sensor_readouts(self, sensor_id, params=None): + return self._request("GET", f"/api/sensors/{sensor_id}/readouts", params=params) + + def sensor_readouts_hourly(self, sensor_id, params=None): + return self._request( + "GET", f"/api/sensors/{sensor_id}/readouts/hourly", params=params + ) diff --git a/server/templates/docs.html b/server/templates/docs.html index cbd0d09..2c45786 100644 --- a/server/templates/docs.html +++ b/server/templates/docs.html @@ -18,7 +18,8 @@

Sensor Readouts — API Documentation

Base URL

-

http://localhost:5000/api

+

{{ main_api_url }}

+

The main app proxies all requests to the sensors API service.

GET /api/sensors

@@ -26,7 +27,7 @@ GET/api/sensors

Return a list of all sensors ordered by name.

-
curl http://localhost:5000/api/sensors
+
curl {{ main_api_url }}/sensors
[
   { "id": 1, "name": "outdoor-station" },
@@ -39,7 +40,7 @@
         POST/api/sensors
         

Create a new sensor.

-
curl -X POST http://localhost:5000/api/sensors \
+
curl -X POST {{ main_api_url }}/sensors \
      -H "Content-Type: application/json" \
      -d '{"name": "new-sensor"}'
@@ -58,7 +59,7 @@ GET/api/sensors/:id/types

Return distinct readout types available for a given sensor.

-
curl http://localhost:5000/api/sensors/1/types
+
curl {{ main_api_url }}/sensors/1/types
["humidity", "pressure", "temperature"]

Errors:

@@ -80,10 +81,10 @@
# all types
-curl http://localhost:5000/api/sensors/1/readouts
+curl {{ main_api_url }}/sensors/1/readouts
 
 # filter by type
-curl "http://localhost:5000/api/sensors/1/readouts?type=temperature"
+curl "{{ main_api_url }}/sensors/1/readouts?type=temperature"
[
   { "value": 24.31, "timestamp": "2026-09-11T10:30:00" },
@@ -96,7 +97,7 @@ curl "http://localhost:5000/api/sensors/1/readouts?type=temperature"
POST/api/sensors/:id/readouts

Add a new readout to an existing sensor. Timestamp is generated server-side.

-
curl -X POST http://localhost:5000/api/sensors/1/readouts \
+
curl -X POST {{ main_api_url }}/sensors/1/readouts \
      -H "Content-Type: application/json" \
      -d '{"type": "temperature", "value": 22.5}'
diff --git a/server/tests/test_split_apps.py b/server/tests/test_split_apps.py new file mode 100644 index 0000000..fb9a522 --- /dev/null +++ b/server/tests/test_split_apps.py @@ -0,0 +1,200 @@ +import os +import tempfile +import threading +import unittest +from pathlib import Path + +from sqlalchemy import inspect +from werkzeug.serving import make_server + +ENV_KEYS = ( + "SENSORS_DATABASE_URL", + "USERS_DATABASE_URL", + "SECRET_KEY", + "SENSORS_API_URL", +) + + +def _save_env(keys): + return {key: os.environ.get(key) for key in keys} + + +def _restore_env(saved): + for key, value in saved.items(): + if value is None: + os.environ.pop(key, None) + else: + os.environ[key] = value + + +class SensorApiTest(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.tmpdir = tempfile.TemporaryDirectory() + cls._env = _save_env(ENV_KEYS) + os.environ["SENSORS_DATABASE_URL"] = ( + f"sqlite:///{Path(cls.tmpdir.name) / 'sensors.db'}" + ) + + from models import db + from sensors_app import create_app + + cls.app = create_app() + with cls.app.app_context(): + db.create_all() + + cls.client = cls.app.test_client() + + @classmethod + def tearDownClass(cls): + from models import db + + with cls.app.app_context(): + db.session.remove() + db.drop_all() + cls.tmpdir.cleanup() + _restore_env(cls._env) + + def test_sensors_db_has_no_user_table(self): + from models import db + + with self.app.app_context(): + tables = set(inspect(db.engine).get_table_names()) + self.assertEqual(tables, {"sensor", "sensor_readout"}) + + def test_all_api_endpoints_are_public(self): + created = self.client.post("/api/sensors", json={"name": "api-test-sensor"}) + self.assertEqual(created.status_code, 201) + sensor_id = created.get_json()["id"] + + readout = self.client.post( + f"/api/sensors/{sensor_id}/readouts", + json={"type": "temperature", "value": 20.5}, + ) + self.assertEqual(readout.status_code, 201) + + self.assertEqual(self.client.get("/api/sensors").status_code, 200) + types = self.client.get(f"/api/sensors/{sensor_id}/types") + self.assertEqual(types.status_code, 200) + self.assertEqual(types.get_json(), ["temperature"]) + + readouts = self.client.get(f"/api/sensors/{sensor_id}/readouts") + self.assertEqual(readouts.status_code, 200) + self.assertEqual(readouts.get_json()[0]["value"], 20.5) + + hourly = self.client.get( + f"/api/sensors/{sensor_id}/readouts/hourly?type=temperature&range=all" + ) + self.assertEqual(hourly.status_code, 200) + self.assertEqual(len(hourly.get_json()), 1) + + +class MainAppTest(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.tmpdir = tempfile.TemporaryDirectory() + cls._env = _save_env(ENV_KEYS) + base = Path(cls.tmpdir.name) + os.environ["SENSORS_DATABASE_URL"] = f"sqlite:///{base / 'sensors.db'}" + os.environ["USERS_DATABASE_URL"] = f"sqlite:///{base / 'main_app.db'}" + os.environ["SECRET_KEY"] = "test-secret" + + from models import db + from sensors_app import create_app as create_sensors_app + from user_models import User, user_db + + cls.sensors_app = create_sensors_app() + with cls.sensors_app.app_context(): + db.create_all() + + cls.server = make_server("127.0.0.1", 0, cls.sensors_app) + cls.thread = threading.Thread(target=cls.server.serve_forever, daemon=True) + cls.thread.start() + os.environ["SENSORS_API_URL"] = f"http://127.0.0.1:{cls.server.server_port}" + + from app import create_app as create_main_app + + cls.main_app = create_main_app() + with cls.main_app.app_context(): + user_db.create_all() + user = User(username="tester") + user.set_password("secret") + user_db.session.add(user) + user_db.session.commit() + + def setUp(self): + self.client = self.main_app.test_client() + + @classmethod + def tearDownClass(cls): + from models import db + from user_models import user_db + + cls.server.shutdown() + cls.thread.join() + with cls.sensors_app.app_context(): + db.session.remove() + db.drop_all() + with cls.main_app.app_context(): + user_db.session.remove() + user_db.drop_all() + cls.tmpdir.cleanup() + _restore_env(cls._env) + + def login(self): + return self.client.post( + "/login", data={"username": "tester", "password": "secret"} + ) + + def create_sensor_and_readout(self, name): + created = self.client.post("/api/sensors", json={"name": name}) + self.assertEqual(created.status_code, 201) + sensor_id = created.get_json()["id"] + + readout = self.client.post( + f"/api/sensors/{sensor_id}/readouts", + json={"type": "temperature", "value": 21.5}, + ) + self.assertEqual(readout.status_code, 201) + return sensor_id + + def test_views_and_login_are_owned_by_main_app(self): + response = self.client.get("/sensors") + self.assertEqual(response.status_code, 302) + self.assertIn("/login", response.headers["Location"]) + + self.assertEqual(self.client.get("/login").status_code, 200) + self.assertEqual(self.login().status_code, 302) + self.assertEqual(self.client.get("/home").status_code, 200) + self.assertEqual(self.client.get("/sensors").status_code, 200) + self.assertEqual(self.client.get("/api/docs").status_code, 200) + + def test_public_device_api_is_proxied_to_sensors(self): + sensor_id = self.create_sensor_and_readout("proxy-test") + + listed = self.client.get("/api/sensors").get_json() + self.assertIn({"id": sensor_id, "name": "proxy-test"}, listed) + + def test_dashboard_reads_require_login_and_are_proxied(self): + sensor_id = self.create_sensor_and_readout("dashboard-test") + + self.assertEqual(self.client.get(f"/api/sensors/{sensor_id}/types").status_code, 401) + + self.login() + types = self.client.get(f"/api/sensors/{sensor_id}/types") + self.assertEqual(types.status_code, 200) + self.assertEqual(types.get_json(), ["temperature"]) + + readouts = self.client.get(f"/api/sensors/{sensor_id}/readouts?type=temperature") + self.assertEqual(readouts.status_code, 200) + self.assertEqual(readouts.get_json()[0]["value"], 21.5) + + hourly = self.client.get( + f"/api/sensors/{sensor_id}/readouts/hourly?type=temperature&range=all" + ) + self.assertEqual(hourly.status_code, 200) + self.assertEqual(len(hourly.get_json()), 1) + + +if __name__ == "__main__": + unittest.main() diff --git a/server/user_models.py b/server/user_models.py new file mode 100644 index 0000000..6ee334c --- /dev/null +++ b/server/user_models.py @@ -0,0 +1,25 @@ +from datetime import datetime, timezone + +from flask_login import UserMixin +from flask_sqlalchemy import SQLAlchemy +from werkzeug.security import check_password_hash, generate_password_hash + +user_db = SQLAlchemy() + + +class User(UserMixin, user_db.Model): + __tablename__ = "user" + + id = user_db.Column(user_db.Integer, primary_key=True) + username = user_db.Column(user_db.String(80), nullable=False, unique=True) + password_hash = user_db.Column(user_db.String(255), nullable=False) + is_admin = user_db.Column(user_db.Boolean, nullable=False, default=False) + created_at = user_db.Column( + user_db.DateTime, default=lambda: datetime.now(timezone.utc) + ) + + def set_password(self, password: str) -> None: + self.password_hash = generate_password_hash(password) + + def check_password(self, password: str) -> bool: + return check_password_hash(self.password_hash, password) diff --git a/server/vps_ssh_tunnel.sh b/server/vps_ssh_tunnel.sh index 99c4073..0ca32e9 100755 --- a/server/vps_ssh_tunnel.sh +++ b/server/vps_ssh_tunnel.sh @@ -1 +1 @@ -ssh -R 5432:localhost:8000 vps +ssh -R 5432:localhost:8001 vps diff --git a/server/yggdrasil-sensors.service b/server/yggdrasil-sensors.service new file mode 100644 index 0000000..702b461 --- /dev/null +++ b/server/yggdrasil-sensors.service @@ -0,0 +1,18 @@ +[Unit] +Description=Yggdrasil Sensors App +After=network.target yggdrasil.service + +[Service] +User=mateusz +Group=mateusz +WorkingDirectory=/home/mateusz/prog/yggdrasil/server +# The sensors API is open to local users and listens on localhost only. +# Data lives in sensors.db; users live in the main app's main_app.db. +#Environment=SENSORS_DB_PATH=/home/mateusz/prog/yggdrasil/server/sensors.db +#Environment=HOST=0.0.0.0 +ExecStart=/home/mateusz/prog/yggdrasil/server/run-prod.sh sensors 8001 +Restart=on-failure +RestartSec=5 + +[Install] +WantedBy=multi-user.target diff --git a/server/yggdrasil.service b/server/yggdrasil.service index 3628ae3..4c385b2 100644 --- a/server/yggdrasil.service +++ b/server/yggdrasil.service @@ -1,12 +1,15 @@ [Unit] -Description=Yggdrasil Sensor API +Description=Yggdrasil Main App After=network.target [Service] User=mateusz Group=mateusz WorkingDirectory=/home/mateusz/prog/yggdrasil/server -ExecStart=/home/mateusz/prog/yggdrasil/server/.venv/bin/gunicorn -c gunicorn.conf.py "app:create_app()" +# Override when the sensors API is not on localhost:8001: +#Environment=SENSORS_API_URL=http://localhost:8001 +#Environment=USERS_DB_PATH=/home/mateusz/prog/yggdrasil/server/main_app.db +ExecStart=/home/mateusz/prog/yggdrasil/server/run-prod.sh main 8000 Restart=on-failure RestartSec=5