sensors is separate API with td

This commit is contained in:
2026-11-01 12:07:04 +01:00
parent b7048b401d
commit 85179c8a80
24 changed files with 1229 additions and 321 deletions
+256 -69
View File
@@ -1,13 +1,26 @@
# Sensor Readouts # Sensor Readouts
A Flask web application for managing and visualizing sensor data. It provides a REST API for creating sensors and storing readouts, along with a frontend dashboard that displays real-time charts and tabular data. Two Flask services with separate responsibilities and separate databases:
- **main app** (`app.py`) — all user-facing views: login, home, API docs and
the `/sensors` dashboard. It owns the **users** database (`main_app.db`) and
proxies data requests to the sensors API.
- **sensors app** (`sensors_app.py`) — API only. It owns the **sensors**
database (`sensors.db`, tables `sensor` and `sensor_readout`) and has no
knowledge of users. Its API is open to every user on the local machine.
```
browser ──> main app :8000 ──HTTP──> sensors API :8001
│ │
main_app.db sensors.db
```
## Tech Stack ## Tech Stack
- **Python 3.13+** - **Python 3.13+**
- **Flask 3** — web framework - **Flask 3** — web framework
- **SQLAlchemy** (via Flask-SQLAlchemy) — ORM - **SQLAlchemy** (via Flask-SQLAlchemy) — ORM
- **SQLite** — database - **SQLite** — databases
- **Alembic** — database migrations - **Alembic** — database migrations
- **Chart.js** — frontend charting library - **Chart.js** — frontend charting library
@@ -15,18 +28,31 @@ A Flask web application for managing and visualizing sensor data. It provides a
``` ```
server/ server/
├── app.py # Flask application factory and API routes ├── app.py # Main app: views + API proxy
├── models.py # SQLAlchemy models (Sensor, SensorReadout) ├── sensors_app.py # Sensors app: API + sensors database
├── seed.py # Database seeder with sample data ├── sensors_client.py # HTTP client used by the main app
├── requirements.txt # Python dependencies ├── auth.py # Flask-Login setup
├── data.db # SQLite database (auto-created) ├── config.py # Config (databases, secret key, sensors URL)
├── alembic.ini # Alembic configuration ├── models.py # Sensor, SensorReadout (sensors DB)
├── alembic/ # Migration scripts ├── user_models.py # User (main_app.db)
│ ├── env.py ├── seed.py # Sample sensor data
│ └── versions/ ├── migrate_legacy_db.py # Import old combined data.db into split DBs
├── run.sh # Development launcher (main/sensors + port)
├── run-prod.sh # Production launcher (gunicorn)
├── gunicorn.conf.py # Gunicorn config (bind from BIND/PORT)
├── alembic.ini # Sensors DB migrations
├── alembic/ # Sensors DB migration scripts
├── alembic_users.ini # Users DB migrations
├── alembic_users/ # Users DB migration scripts
├── sensors.db # Sensors database (sensors app)
├── main_app.db # Users database (main app)
├── data.db # Legacy combined DB (not used by default)
├── requirements.txt
└── templates/ └── templates/
├── sensors.html # Dashboard with chart and table ├── home.html
└── docs.html # API documentation page ├── login.html
├── docs.html
└── sensors.html
``` ```
## Setup ## Setup
@@ -39,20 +65,50 @@ source .venv/bin/activate
# Install dependencies # Install dependencies
pip install -r requirements.txt pip install -r requirements.txt
# Run database migrations # Sensors database
alembic upgrade head alembic upgrade head
# (Optional) Seed the database with sample data # Users database
alembic -c alembic_users.ini upgrade head
# (Optional) Sample sensor data
python seed.py python seed.py
``` ```
The `sensors.db` and `main_app.db` files are created by the migrations and are
ignored by git (`*.db`).
### Migrating the old combined database
Previous versions stored users, sensors and readouts in `data.db`. The new
layout uses `sensors.db` and `main_app.db`. To copy the old rows (the legacy file
is only read, never modified):
```bash
alembic upgrade head
alembic -c alembic_users.ini upgrade head
python migrate_legacy_db.py
```
For custom paths use `--legacy`, `--sensors` and `--main-app`, for example:
```bash
python migrate_legacy_db.py --legacy old.db --sensors /data/sensors.db \
--main-app /data/main_app.db
```
After verifying the new databases you can archive or remove `data.db`.
## Users ## Users
There is no public registration. All accounts are created by an administrator. User accounts live only in `main_app.db` (main app). There is no public
Pages (`/`, `/home`, `/sensors`, `/api/docs`) and dashboard data endpoints registration; accounts are created by an administrator.
(`GET /api/sensors/:id/types`, `/readouts`, `/readouts/hourly`) require login.
`GET /api/sensors` (list, needed by devices to resolve name to id) and Main app pages (`/`, `/home`, `/api/docs`, `/sensors`) and dashboard read
`POST /api/*` stay open so devices can push data without a browser session. endpoints (`GET /api/sensors/:id/types`, `/readouts`, `/readouts/hourly`)
require login. Device endpoints (`GET/POST /api/sensors`,
`POST /api/sensors/:id/readouts`) stay open so devices can push data without a
browser session.
Set a secret key for sessions (required in production): Set a secret key for sessions (required in production):
@@ -60,15 +116,12 @@ Set a secret key for sessions (required in production):
export SECRET_KEY='long-random-value' export SECRET_KEY='long-random-value'
``` ```
If `SECRET_KEY` is unset, the main app falls back to `.secret_key` in this
directory (created automatically by `run-prod.sh`).
### First admin ### First admin
The database starts with an empty `user` table. Create the first admin
with the CLI command (flags, env vars, or interactive prompts):
```bash ```bash
# Run database migrations first
alembic upgrade head
# Option 1: flags # Option 1: flags
flask --app app create-admin --username admin --password 'secret' flask --app app create-admin --username admin --password 'secret'
@@ -79,47 +132,95 @@ ADMIN_USERNAME=admin ADMIN_PASSWORD=secret flask --app app create-admin
flask --app app create-admin flask --app app create-admin
``` ```
Running `create-admin` for an existing username resets its password and Running `create-admin` for an existing username resets its password and grants
grants `is_admin=True`. After that log in at `/login` — you land on `/home`. `is_admin=True`. Log in at `/login`; you land on `/home`.
## Running ## Running
The first argument selects the app (`main` or `sensors`), the second the port.
The main app (`:8000`) listens on all interfaces; the sensors API (`:8001`)
listens on `127.0.0.1` by default and is open to every local user without
authentication.
### Development ### Development
```bash ```bash
python app.py ./run.sh # main app on 8000
./run.sh main 8000
./run.sh sensors 8001
``` ```
The server starts on `http://localhost:5000`.
### Production ### Production
```bash ```bash
# Install dependencies (includes gunicorn)
pip install -r requirements.txt pip install -r requirements.txt
# Run with gunicorn ./run-prod.sh main 8000
gunicorn -c gunicorn.conf.py "app:create_app()" ./run-prod.sh sensors 8001
``` ```
Gunicorn starts on `http://0.0.0.0:8000` with `cpu * 2 + 1` workers. Both scripts use `.venv` when present. `run-prod.sh` runs the matching
migrations before starting each service.
#### systemd service To expose the sensors API on the network:
```bash ```bash
sudo cp yggdrasil.service /etc/systemd/system/ HOST=0.0.0.0 ./run.sh sensors 8001
sudo systemctl daemon-reload # or
sudo systemctl enable --now yggdrasil BIND=0.0.0.0:8001 ./run-prod.sh sensors 8001
``` ```
| Route | Description | ### Configuration
|---|---|
| `/sensors` | Dashboard with interactive chart and data table |
| `/api/docs` | API documentation with curl examples |
## Database | Variable | Default | Purpose |
|---|---|---|
| `SENSORS_API_URL` | `http://localhost:8001` | Sensors API address used by the main app |
| `SENSORS_DATABASE_URL` | `sqlite:///<server>/sensors.db` | Sensors database URL |
| `SENSORS_DB_PATH` | `<server>/sensors.db` | SQLite path when URL is unset |
| `USERS_DATABASE_URL` | `sqlite:///<server>/main_app.db` | Users database URL |
| `USERS_DB_PATH` | `<server>/main_app.db` | SQLite path when URL is unset |
| `SECRET_KEY` | `.secret_key` file | Main app session signing key |
| `HOST` | main `0.0.0.0`, sensors `127.0.0.1` | Listen address |
| `BIND` | `<HOST>:<port>` | Gunicorn bind override |
Two tables are used: `DATABASE_URL`/`DB_PATH` are accepted as legacy aliases for the sensors DB.
When the services run on different machines, point `SENSORS_API_URL` at the
address reachable from the main app process. For example, with the sensors API
on your laptop (port 8001) and the main app on a VPS:
```bash
# On the laptop: expose the local sensors API as VPS port 5432
ssh -R 5432:localhost:8001 vps
# On the VPS
SENSORS_API_URL=http://localhost:5432 ./run-prod.sh main 8000
```
### systemd services
```bash
sudo cp yggdrasil.service yggdrasil-sensors.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now yggdrasil yggdrasil-sensors
```
| Service | Port | Routes |
|---|---|---|
| main | 8000 | `/`, `/login`, `/home`, `/sensors`, `/api/docs`, `/api/*` (proxy) |
| sensors | 8001 | `/api/*` (sensors and readouts) |
### Tests
```bash
python -m unittest discover -s tests -v
```
## Databases
### sensors.db
Owned by the sensors app. Contains only sensor data.
**sensor** **sensor**
| Column | Type | Description | | Column | Type | Description |
@@ -137,64 +238,150 @@ Two tables are used:
| `value` | Float | Measured value | | `value` | Float | Measured value |
| `timestamp` | DateTime | Readout timestamp (UTC) | | `timestamp` | DateTime | Readout timestamp (UTC) |
### main_app.db (users)
Owned by the main app. Contains only accounts.
**user**
| Column | Type | Description |
|---|---|---|
| `id` | Integer | Primary key |
| `username` | String(80) | Unique username |
| `password_hash` | String(255) | Password hash |
| `is_admin` | Boolean | Administrator flag |
| `created_at` | DateTime | Creation timestamp (UTC) |
### Inspecting the databases on Linux
The databases are ordinary SQLite files. Use the `sqlite3` CLI (install it with
`sudo apt install sqlite3` on Debian/Ubuntu):
```bash
# List tables
sqlite3 sensors.db ".tables"
sqlite3 main_app.db ".tables"
# Show the schema (all tables or one table)
sqlite3 sensors.db ".schema"
sqlite3 sensors.db ".schema sensor_readout"
# Show columns of one table
sqlite3 sensors.db "PRAGMA table_info(sensor);"
# Inspect rows (readable column output)
sqlite3 -header -column sensors.db \
"SELECT id, name, created_at FROM sensor LIMIT 10;"
sqlite3 -header -column sensors.db \
"SELECT id, sensor_id, type, value, timestamp FROM sensor_readout
ORDER BY timestamp DESC LIMIT 10;"
# Users (do not print password_hash unnecessarily)
sqlite3 -header -column main_app.db \
"SELECT id, username, is_admin, created_at FROM user;"
# Counts and migration version
sqlite3 sensors.db "SELECT COUNT(*) FROM sensor_readout;"
sqlite3 sensors.db "SELECT * FROM alembic_version;"
sqlite3 main_app.db "SELECT * FROM alembic_version;"
```
Inside an interactive `sqlite3` session you can also use:
```sql
.tables -- list tables
.headers on -- show column names
.mode column -- align columns
.schema sensor -- show CREATE TABLE
SELECT * FROM sensor; -- run a query
.quit -- exit
```
For read-only inspection add `-readonly`:
```bash
sqlite3 -readonly -header -column sensors.db \
"SELECT * FROM sensor_readout LIMIT 5;"
```
If `sqlite3` is not available, the same tables can be listed with Python:
```bash
.venv/bin/python - <<'PY'
import sqlite3
con = sqlite3.connect("sensors.db")
print([row[0] for row in con.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
)])
PY
```
### Migrations ### Migrations
```bash ```bash
# Create a new migration after model changes # Sensors DB
alembic upgrade head
alembic revision --autogenerate -m "description" alembic revision --autogenerate -m "description"
# Apply pending migrations # Users DB
alembic upgrade head alembic -c alembic_users.ini upgrade head
alembic -c alembic_users.ini revision --autogenerate -m "description"
# Roll back one step
alembic downgrade -1
``` ```
## API ## API
Base URL: `http://localhost:5000/api` Public API (main app): `http://localhost:8000/api`
Sensors API (local machine): `http://localhost:8001/api`
The main app proxies every `/api/sensors*` request to the sensors API, so
clients and devices can keep using port 8000.
### Endpoints ### Endpoints
| Method | Path | Description | | Method | Path | Auth | Description |
|---|---|---| |---|---|---|---|
| `GET` | `/api/sensors` | List all sensors | | `GET` | `/api/sensors` | public | List all sensors |
| `POST` | `/api/sensors` | Create a new sensor | | `POST` | `/api/sensors` | public | Create a new sensor |
| `GET` | `/api/sensors/:id/types` | List readout types for a sensor | | `GET` | `/api/sensors/:id/types` | login | List readout types for a sensor |
| `GET` | `/api/sensors/:id/readouts` | Latest readouts (max 300, filterable by `?type=`) | | `GET` | `/api/sensors/:id/readouts` | login | Latest readouts (max 300, filterable by `?type=`) |
| `GET` | `/api/sensors/:id/readouts/hourly` | Hourly averaged readouts (filterable by `?type=`) | | `GET` | `/api/sensors/:id/readouts/hourly` | login | Hourly averaged readouts |
| `POST` | `/api/sensors/:id/readouts` | Add a new readout | | `POST` | `/api/sensors/:id/readouts` | public | Add a new readout |
Calling the sensors API directly on `:8001` skips login; it is open to every
local user by design.
### Examples ### Examples
```bash ```bash
# List sensors # List sensors
curl http://localhost:5000/api/sensors curl http://localhost:8000/api/sensors
# Create a sensor # Create a sensor
curl -X POST http://localhost:5000/api/sensors \ curl -X POST http://localhost:8000/api/sensors \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{"name": "bmp280"}' -d '{"name": "bmp280"}'
# Get temperature readouts for sensor 1 # Get temperature readouts for sensor 1 (requires a logged-in session)
curl "http://localhost:5000/api/sensors/1/readouts?type=temperature" curl -b cookies.txt "http://localhost:8000/api/sensors/1/readouts?type=temperature"
# Get hourly averaged data # Get hourly averaged data (requires a logged-in session)
curl "http://localhost:5000/api/sensors/1/readouts/hourly?type=temperature" curl -b cookies.txt "http://localhost:8000/api/sensors/1/readouts/hourly?type=temperature"
# Add a readout # Add a readout
curl -X POST http://localhost:5000/api/sensors/1/readouts \ curl -X POST http://localhost:8000/api/sensors/1/readouts \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{"type": "temperature", "value": 22.5}' -d '{"type": "temperature", "value": 22.5}'
# Direct local access with no login
curl http://localhost:8001/api/sensors/1/readouts
``` ```
Full API documentation with request/response examples is available at `/api/docs`. Full API documentation with request/response examples is available at `/api/docs`.
## Frontend ## Frontend
The dashboard at `/sensors` provides: The dashboard in the main app at `/sensors` provides:
- **Sensor dropdown** — select a sensor from the list - **Sensor dropdown** — select a sensor from the list
- **Type dropdown** — select a readout type (populated based on the chosen sensor) - **Type dropdown** — select a readout type (populated based on the chosen sensor)
- **Line chart** — hourly averaged values rendered with Chart.js - **Line chart** — raw or hourly-averaged values for the selected range
- **Data table** — raw readouts with formatted timestamps (latest 300 entries) - **Data table** — raw readouts with formatted timestamps (latest 300 entries)
+1 -1
View File
@@ -8,7 +8,7 @@ from sqlalchemy import engine_from_config, pool
# Add the project root to sys.path so that models can be imported. # Add the project root to sys.path so that models can be imported.
sys.path.insert(0, os.path.realpath(os.path.join(os.path.dirname(__file__), ".."))) sys.path.insert(0, os.path.realpath(os.path.join(os.path.dirname(__file__), "..")))
from app import create_app from sensors_app import create_app
from models import db from models import db
# Create the Flask application to get the database URI. # Create the Flask application to get the database URI.
@@ -1,4 +1,4 @@
"""add user table """deprecated user table (users moved to the main app)
Revision ID: c59e689377c2 Revision ID: c59e689377c2
Revises: 7f92bfc47dfc Revises: 7f92bfc47dfc
@@ -7,9 +7,6 @@ Create Date: 2026-10-03 19:06:44.095699
""" """
from typing import Sequence, Union from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic. # revision identifiers, used by Alembic.
revision: str = 'c59e689377c2' revision: str = 'c59e689377c2'
@@ -19,22 +16,12 @@ depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None: def upgrade() -> None:
"""Upgrade schema.""" """No-op: the sensors database no longer stores users.
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('user', The revision is kept so legacy databases that already point at it can
sa.Column('id', sa.Integer(), nullable=False), still be upgraded. New sensors databases create only sensor tables.
sa.Column('username', sa.String(length=80), nullable=False), """
sa.Column('password_hash', sa.String(length=255), nullable=False),
sa.Column('is_admin', sa.Boolean(), nullable=False),
sa.Column('created_at', sa.DateTime(), nullable=True),
sa.PrimaryKeyConstraint('id'),
sa.UniqueConstraint('username')
)
# ### end Alembic commands ###
def downgrade() -> None: def downgrade() -> None:
"""Downgrade schema.""" """No-op: the user table is no longer part of the sensors database."""
# ### commands auto generated by Alembic - please adjust! ###
op.drop_table('user')
# ### end Alembic commands ###
+37
View File
@@ -0,0 +1,37 @@
[alembic]
script_location = %(here)s/alembic_users
prepend_sys_path = .
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
+52
View File
@@ -0,0 +1,52 @@
import os
import sys
from logging.config import fileConfig
from alembic import context
from sqlalchemy import engine_from_config, pool
sys.path.insert(0, os.path.realpath(os.path.join(os.path.dirname(__file__), "..")))
from config import users_database_uri
from user_models import user_db
config = context.config
config.set_main_option("sqlalchemy.url", users_database_uri())
if config.config_file_name is not None:
fileConfig(config.config_file_name)
target_metadata = user_db.metadata
def run_migrations_offline() -> None:
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
+28
View File
@@ -0,0 +1,28 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
# revision identifiers, used by Alembic.
revision: str = ${repr(up_revision)}
down_revision: Union[str, Sequence[str], None] = ${repr(down_revision)}
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
def upgrade() -> None:
"""Upgrade schema."""
${upgrades if upgrades else "pass"}
def downgrade() -> None:
"""Downgrade schema."""
${downgrades if downgrades else "pass"}
@@ -0,0 +1,35 @@
"""create user table
Revision ID: 1a7c3e5d9b21
Revises:
Create Date: 2026-11-01 12:00:00.000000
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = "1a7c3e5d9b21"
down_revision: Union[str, Sequence[str], None] = None
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"user",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("username", sa.String(length=80), nullable=False),
sa.Column("password_hash", sa.String(length=255), nullable=False),
sa.Column("is_admin", sa.Boolean(), nullable=False),
sa.Column("created_at", sa.DateTime(), nullable=True),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("username"),
)
def downgrade() -> None:
op.drop_table("user")
+64 -182
View File
@@ -1,52 +1,61 @@
import os import os
from datetime import datetime, timedelta, timezone
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
import click import click
from flask import Flask, jsonify, redirect, render_template, request, url_for from flask import (
from flask_login import ( Flask,
LoginManager, jsonify,
current_user, redirect,
login_required, render_template,
login_user, request,
logout_user, url_for,
) )
from flask_login import current_user, login_required, login_user, logout_user
from models import Sensor, SensorReadout, User, db from auth import init_login_manager
from config import load_secret_key, sensors_api_url, users_database_uri
login_manager = LoginManager() from sensors_client import SensorsAPIError, SensorsClient
from user_models import User, user_db
def create_app(): def create_app():
app = Flask(__name__) app = Flask(__name__)
db_path = os.path.join(os.path.dirname(__file__), "data.db") app.config["SECRET_KEY"] = load_secret_key()
app.config["SQLALCHEMY_DATABASE_URI"] = f"sqlite:///{db_path}" app.config["SQLALCHEMY_DATABASE_URI"] = users_database_uri()
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
app.config["SECRET_KEY"] = os.environ.get("SECRET_KEY", "dev-secret-key-change-me") app.config["SENSORS_API_URL"] = sensors_api_url()
db.init_app(app) user_db.init_app(app)
login_manager.init_app(app) sensors = SensorsClient(app.config["SENSORS_API_URL"])
login_manager.login_view = "login"
@login_manager.user_loader def _login_redirect(req):
def load_user(user_id):
return db.session.get(User, int(user_id))
@login_manager.unauthorized_handler
def unauthorized():
# Return JSON for API calls instead of redirecting to the login page.
if request.path.startswith("/api/"):
return jsonify({"error": "authentication required"}), 401
next_page = request.path
login_url = url_for("login") login_url = url_for("login")
if next_page and next_page != login_url: if req.path and req.path != login_url:
login_url += f"?next={next_page}" return url_for("login", next=req.path)
return redirect(login_url) return login_url
def _load_user(user_id):
try:
return user_db.session.get(User, int(user_id))
except (TypeError, ValueError):
return None
init_login_manager(app, _login_redirect, _load_user)
def _is_safe_next(target: str | None) -> bool: def _is_safe_next(target: str | None) -> bool:
return bool(target) and target.startswith("/") and not target.startswith("//") return bool(target) and target.startswith("/") and not target.startswith("//")
def _proxy(call):
try:
status, data = call()
except SensorsAPIError as exc:
return jsonify({"error": str(exc)}), 502
return jsonify(data), status
@app.context_processor
def inject_service_urls():
return {"main_api_url": request.url_root.rstrip("/") + "/api"}
# --- Pages --- # --- Pages ---
@app.route("/") @app.route("/")
@@ -94,176 +103,49 @@ def create_app():
def api_docs(): def api_docs():
return render_template("docs.html", user=current_user) return render_template("docs.html", user=current_user)
# --- API: device endpoints (intentionally public) --- # --- API: proxied to the sensors service ---
# The sensor firmware has no browser session, but it needs the sensor # The browser only talks to this app; the sensors service owns the
# list to resolve a name to an id before pushing readouts. # sensor database and the actual API implementation.
@app.route("/api/sensors") @app.route("/api/sensors")
def api_sensors(): def api_sensors():
sensors = Sensor.query.order_by(Sensor.name).all() return _proxy(sensors.list_sensors)
return jsonify([{"id": s.id, "name": s.name} for s in sensors])
# --- API: dashboard reads (login required, the browser session cookie is enough) --- @app.route("/api/sensors", methods=["POST"])
def api_create_sensor():
return _proxy(lambda: sensors.create_sensor(request.get_json(silent=True)))
@app.route("/api/sensors/<int:sensor_id>/readouts", methods=["POST"])
def api_create_readout(sensor_id):
return _proxy(
lambda: sensors.create_readout(sensor_id, request.get_json(silent=True))
)
@app.route("/api/sensors/<int:sensor_id>/types") @app.route("/api/sensors/<int:sensor_id>/types")
@login_required @login_required
def api_sensor_types(sensor_id): def api_sensor_types(sensor_id):
types = ( return _proxy(lambda: sensors.sensor_types(sensor_id))
db.session.query(SensorReadout.type)
.filter_by(sensor_id=sensor_id)
.distinct()
.order_by(SensorReadout.type)
.all()
)
return jsonify([t[0] for t in types])
@app.route("/api/sensors/<int:sensor_id>/readouts") @app.route("/api/sensors/<int:sensor_id>/readouts")
@login_required @login_required
def api_sensor_readouts(sensor_id): def api_sensor_readouts(sensor_id):
type_ = request.args.get("type") return _proxy(
query = SensorReadout.query.filter_by(sensor_id=sensor_id) lambda: sensors.sensor_readouts(
if type_: sensor_id, {"type": request.args.get("type")}
query = query.filter_by(type=type_) )
readouts = query.order_by(SensorReadout.timestamp.desc()).limit(300).all()
return jsonify(
[
{"value": r.value, "timestamp": r.timestamp.isoformat() + "Z"}
for r in readouts
]
) )
@app.route("/api/sensors/<int:sensor_id>/readouts/hourly") @app.route("/api/sensors/<int:sensor_id>/readouts/hourly")
@login_required @login_required
def api_sensor_readouts_hourly(sensor_id): def api_sensor_readouts_hourly(sensor_id):
type_ = request.args.get("type") return _proxy(
range_ = request.args.get("range", "all") lambda: sensors.sensor_readouts_hourly(sensor_id, request.args.to_dict())
# Frontend wysyła strefę czasową przeglądarki (np. Europe/Warsaw),
# żeby "today" znaczyło "od północy czasu lokalnego", a nie UTC.
tz_name = request.args.get("tz", "Europe/Warsaw") or "Europe/Warsaw"
try:
local_tz = ZoneInfo(tz_name)
except (ZoneInfoNotFoundError, ValueError):
return jsonify({"error": f"unknown timezone: {tz_name}"}), 400
# Timestamps are stored as naive UTC, so the cutoff must be naive UTC too.
now_utc = datetime.now(timezone.utc)
now_local = now_utc.astimezone(local_tz)
if range_ == "today":
midnight_local = now_local.replace(hour=0, minute=0, second=0, microsecond=0)
cutoff = midnight_local.astimezone(timezone.utc).replace(tzinfo=None)
elif range_ == "24h":
cutoff = (now_utc - timedelta(hours=24)).replace(tzinfo=None)
elif range_ == "week":
cutoff = (now_utc - timedelta(days=7)).replace(tzinfo=None)
elif range_ == "month":
cutoff = (now_utc - timedelta(days=30)).replace(tzinfo=None)
elif range_ == "all":
cutoff = None
else:
return jsonify({"error": "range must be one of: today, 24h, week, month, all"}), 400
query = db.session.query(SensorReadout.timestamp, SensorReadout.value).filter(
SensorReadout.sensor_id == sensor_id
) )
if cutoff is not None:
query = query.filter(SensorReadout.timestamp >= cutoff)
if type_:
query = query.filter(SensorReadout.type == type_)
rows = query.order_by(SensorReadout.timestamp).all()
# Zakresy today i 24h domyślnie zwracają wszystkie pomiary bez
# uśredniania; checkbox "hourly average" wymusza średnie godzinowe.
averaged = range_ not in ("today", "24h") or request.args.get(
"avg", "0"
).lower() in ("1", "true", "yes")
if not averaged:
return jsonify(
[
{
"timestamp": ts.strftime("%Y-%m-%dT%H:%M:%S"),
"value": value,
}
for ts, value in rows
]
)
# Pozostałe zakresy: średnia godzinowa liczona po pełnej godzinie
# lokalnej (poprawne też przy zmianie DST), znacznik to początek
# godziny w UTC (frontend dokleja "Z").
buckets: dict[datetime, list[float]] = {}
for ts, value in rows:
ts_utc = ts.replace(tzinfo=timezone.utc)
hour_local = ts_utc.astimezone(local_tz).replace(
minute=0, second=0, microsecond=0
)
hour_utc = hour_local.astimezone(timezone.utc).replace(tzinfo=None)
buckets.setdefault(hour_utc, []).append(value)
result = [
{
"timestamp": hour.strftime("%Y-%m-%dT%H:00:00"),
"value": round(sum(values) / len(values), 2),
}
for hour, values in sorted(buckets.items())
]
return jsonify(result)
# --- API: device writes (intentionally public — called by devices/sensors) ---
@app.route("/api/sensors", methods=["POST"])
def api_create_sensor():
data = request.get_json(silent=True)
if not data or "name" not in data:
return jsonify({"error": "name is required"}), 400
name = data["name"].strip()
if not name:
return jsonify({"error": "name cannot be empty"}), 400
if Sensor.query.filter_by(name=name).first():
return jsonify({"error": "sensor already exists"}), 409
sensor = Sensor(name=name)
db.session.add(sensor)
db.session.commit()
return jsonify({"id": sensor.id, "name": sensor.name}), 201
@app.route("/api/sensors/<int:sensor_id>/readouts", methods=["POST"])
def api_create_readout(sensor_id):
sensor = db.session.get(Sensor, sensor_id)
if not sensor:
return jsonify({"error": "sensor not found"}), 404
data = request.get_json(silent=True)
if not data:
return jsonify({"error": "request body is required"}), 400
missing = [f for f in ("type", "value") if f not in data]
if missing:
return jsonify({"error": f"missing fields: {', '.join(missing)}"}), 400
try:
value = float(data["value"])
except (TypeError, ValueError):
return jsonify({"error": "value must be a number"}), 400
readout = SensorReadout(
sensor_id=sensor_id,
type=str(data["type"]).strip(),
value=value,
)
db.session.add(readout)
db.session.commit()
return jsonify(
{
"id": readout.id,
"sensor_id": readout.sensor_id,
"type": readout.type,
"value": readout.value,
"timestamp": readout.timestamp.isoformat() + "Z",
}
), 201
# --- Command for creating the first admin --- # --- Command for creating the first admin ---
# Usage: # Usage:
# flask --app app create-admin --username admin --password 'secret' # flask --app app create-admin --username admin --password 'secret'
# ADMIN_USERNAME=admin ADMIN_PASSWORD=secret flask --app app create-admin # ADMIN_USERNAME=admin ADMIN_PASSWORD=secret flask --app app create-admin
# flask --app app create-admin (asks interactively)
@app.cli.command("create-admin") @app.cli.command("create-admin")
@click.option("--username", default=None, help="Administrator username") @click.option("--username", default=None, help="Administrator username")
@@ -287,13 +169,13 @@ def create_app():
if user: if user:
user.set_password(password) user.set_password(password)
user.is_admin = True user.is_admin = True
db.session.commit() user_db.session.commit()
click.echo(f"Updated existing user '{username}' (granted is_admin=True).") click.echo(f"Updated existing user '{username}' (granted is_admin=True).")
else: else:
user = User(username=username, is_admin=True) user = User(username=username, is_admin=True)
user.set_password(password) user.set_password(password)
db.session.add(user) user_db.session.add(user)
db.session.commit() user_db.session.commit()
click.echo(f"Created administrator '{username}'.") click.echo(f"Created administrator '{username}'.")
return app return app
@@ -301,4 +183,4 @@ def create_app():
if __name__ == "__main__": if __name__ == "__main__":
app = create_app() app = create_app()
app.run(debug=True, host='0.0.0.0') app.run(debug=True, host='0.0.0.0', port=8000)
+17
View File
@@ -0,0 +1,17 @@
from flask import jsonify, redirect, request
from flask_login import LoginManager
def init_login_manager(app, login_url_builder, load_user, api_prefixes=("/api/",)):
login_manager = LoginManager()
login_manager.init_app(app)
login_manager.user_loader(load_user)
@login_manager.unauthorized_handler
def unauthorized():
# API clients get JSON instead of a redirect to the login page.
if request.path.startswith(tuple(api_prefixes)):
return jsonify({"error": "authentication required"}), 401
return redirect(login_url_builder(request))
return login_manager
+45
View File
@@ -0,0 +1,45 @@
import os
from pathlib import Path
BASE_DIR = Path(__file__).resolve().parent
SECRET_FILE = BASE_DIR / ".secret_key"
DEFAULT_SENSORS_PORT = 8001
def load_secret_key() -> str:
"""Return a stable key used to sign the main app session cookies."""
key = os.environ.get("SECRET_KEY")
if key:
return key
if SECRET_FILE.exists():
return SECRET_FILE.read_text().strip()
return "dev-secret-key-change-me"
def sensors_database_uri() -> str:
"""Database owned by the sensors API (sensors and readouts only)."""
configured = os.environ.get("SENSORS_DATABASE_URL") or os.environ.get("DATABASE_URL")
if configured:
return configured
db_path = os.environ.get(
"SENSORS_DB_PATH", os.environ.get("DB_PATH", str(BASE_DIR / "sensors.db"))
)
return f"sqlite:///{db_path}"
def users_database_uri() -> str:
"""Database owned by the main app (user accounts only)."""
configured = os.environ.get("USERS_DATABASE_URL")
if configured:
return configured
db_path = os.environ.get("USERS_DB_PATH", str(BASE_DIR / "main_app.db"))
return f"sqlite:///{db_path}"
def sensors_api_url() -> str:
"""Base URL used by the main app to call the sensors API (server-to-server)."""
configured = os.environ.get("SENSORS_API_URL")
if configured:
return configured.rstrip("/")
return f"http://localhost:{DEFAULT_SENSORS_PORT}"
+1 -1
View File
@@ -1,6 +1,6 @@
import os import os
bind = "0.0.0.0:8000" bind = os.environ.get("BIND") or f"0.0.0.0:{os.environ.get('PORT', '8000')}"
workers = min(os.cpu_count() * 2 + 1, 9) workers = min(os.cpu_count() * 2 + 1, 9)
timeout = 30 timeout = 30
keepalive = 5 keepalive = 5
+73
View File
@@ -0,0 +1,73 @@
"""Copy data from a legacy combined data.db into the split databases.
The legacy database contained users, sensors and readouts in one file. Run the
new migrations first, then import the old rows:
alembic upgrade head
alembic -c alembic_users.ini upgrade head
python migrate_legacy_db.py
The legacy file is only read; nothing is deleted from it.
"""
import argparse
from pathlib import Path
from sqlalchemy import MetaData, Table, create_engine, inspect, select
from sqlalchemy.dialects.sqlite import insert
from models import Sensor, SensorReadout
from user_models import User
def _sqlite_engine(path: Path):
if not path.exists():
raise SystemExit(f"database not found: {path}")
return create_engine(f"sqlite:///{path}")
def _copy_table(source, destination, table) -> int:
if table.name not in inspect(source).get_table_names():
return 0
source_table = Table(table.name, MetaData(), autoload_with=source)
with source.connect() as conn:
rows = [dict(row._mapping) for row in conn.execute(select(source_table))]
if not rows:
return 0
with destination.begin() as conn:
conn.execute(insert(table).on_conflict_do_nothing(), rows)
return len(rows)
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--legacy", default="data.db", type=Path)
parser.add_argument("--sensors", default="sensors.db", type=Path)
parser.add_argument("--main-app", default="main_app.db", type=Path)
args = parser.parse_args()
legacy = _sqlite_engine(args.legacy)
sensors = _sqlite_engine(args.sensors)
main_app = _sqlite_engine(args.main_app)
if not inspect(sensors).has_table(Sensor.__tablename__):
raise SystemExit(f"{args.sensors} has no schema; run alembic upgrade head first")
if not inspect(main_app).has_table(User.__tablename__):
raise SystemExit(
f"{args.main_app} has no schema; run alembic -c alembic_users.ini upgrade head first"
)
sensor_count = _copy_table(legacy, sensors, Sensor.__table__)
readout_count = _copy_table(legacy, sensors, SensorReadout.__table__)
user_count = _copy_table(legacy, main_app, User.__table__)
print(
f"Copied {sensor_count} sensors, {readout_count} readouts and "
f"{user_count} users. The legacy database was not modified."
)
if __name__ == "__main__":
main()
-18
View File
@@ -1,28 +1,10 @@
from datetime import datetime, timezone from datetime import datetime, timezone
from flask_login import UserMixin
from flask_sqlalchemy import SQLAlchemy from flask_sqlalchemy import SQLAlchemy
from werkzeug.security import check_password_hash, generate_password_hash
db = SQLAlchemy() db = SQLAlchemy()
class User(UserMixin, db.Model):
__tablename__ = "user"
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), nullable=False, unique=True)
password_hash = db.Column(db.String(255), nullable=False)
is_admin = db.Column(db.Boolean, nullable=False, default=False)
created_at = db.Column(db.DateTime, default=lambda: datetime.now(timezone.utc))
def set_password(self, password: str) -> None:
self.password_hash = generate_password_hash(password)
def check_password(self, password: str) -> bool:
return check_password_hash(self.password_hash, password)
class Sensor(db.Model): class Sensor(db.Model):
__tablename__ = "sensor" __tablename__ = "sensor"
+56 -19
View File
@@ -1,35 +1,68 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Production startup (gunicorn) with a stable SECRET_KEY. # Production startup (gunicorn). Pick the service and port:
# #
# Key resolution order: # ./run-prod.sh # main app on 8000
# ./run-prod.sh main 8000 # main app on 8000
# ./run-prod.sh sensors 8001 # sensors app on 8001 (localhost only)
#
# SECRET_KEY resolution (main app sessions):
# 1. SECRET_KEY environment variable (takes precedence), # 1. SECRET_KEY environment variable (takes precedence),
# 2. .secret_key file next to this script (created once on first start), # 2. .secret_key file next to this script (created once on first start),
# 3. otherwise a random key is generated and saved to .secret_key. # 3. otherwise a random key is generated and saved to .secret_key.
# #
# A stable key matters: it signs the session cookies, so changing it # The sensors API is open to every user on the local machine. To expose it
# on every restart would log out all users. # on the network explicitly set HOST=0.0.0.0 (or BIND).
set -euo pipefail set -euo pipefail
cd "$(dirname "$0")" cd "$(dirname "$0")"
SECRET_FILE=".secret_key" APP="${1:-${APP:-main}}"
PORT="${2:-${PORT:-}}"
if [ -z "${SECRET_KEY:-}" ]; then case "$APP" in
if [ -f "$SECRET_FILE" ]; then main)
SECRET_KEY="$(cat "$SECRET_FILE")" MODULE="app:create_app"
else DEFAULT_PORT=8000
SECRET_KEY="$(python3 -c 'import secrets; print(secrets.token_hex(32))')" DEFAULT_HOST="0.0.0.0"
printf '%s' "$SECRET_KEY" > "$SECRET_FILE" ;;
chmod 600 "$SECRET_FILE" sensors)
echo "Generated a new SECRET_KEY, saved to $SECRET_FILE" >&2 MODULE="sensors_app:create_app"
DEFAULT_PORT=8001
DEFAULT_HOST="127.0.0.1"
;;
*)
echo "Usage: $0 [main|sensors] [port]" >&2
exit 2
;;
esac
PORT="${PORT:-$DEFAULT_PORT}"
HOST="${HOST:-$DEFAULT_HOST}"
if [ "$APP" = "main" ]; then
SECRET_FILE=".secret_key"
if [ -z "${SECRET_KEY:-}" ]; then
if [ -f "$SECRET_FILE" ]; then
SECRET_KEY="$(cat "$SECRET_FILE")"
else
SECRET_KEY="$(python3 -c 'import secrets; print(secrets.token_hex(32))')"
printf '%s' "$SECRET_KEY" > "$SECRET_FILE"
chmod 600 "$SECRET_FILE"
echo "Generated a new SECRET_KEY, saved to $SECRET_FILE" >&2
fi
export SECRET_KEY
fi fi
export SECRET_KEY
fi fi
# Migrate before starting, so the schema matches the code. # Each service runs migrations for its own database.
if [ -x ".venv/bin/alembic" ]; then if [ -x ".venv/bin/alembic" ]; then
.venv/bin/alembic upgrade head ALEMBIC=".venv/bin/alembic"
elif command -v alembic >/dev/null 2>&1; then else
alembic upgrade head ALEMBIC="alembic"
fi
if [ "$APP" = "main" ]; then
"$ALEMBIC" -c alembic_users.ini upgrade head
else
"$ALEMBIC" -c alembic.ini upgrade head
fi fi
if [ -x ".venv/bin/gunicorn" ]; then if [ -x ".venv/bin/gunicorn" ]; then
@@ -38,4 +71,8 @@ else
GUNICORN="gunicorn" GUNICORN="gunicorn"
fi fi
exec "$GUNICORN" -c gunicorn.conf.py "app:create_app()" export BIND="${BIND:-$HOST:$PORT}"
echo "Starting $APP app on http://$BIND" >&2
# Gunicorn needs the factory call syntax ("module:create_app()"), otherwise it
# treats create_app itself as the WSGI callable.
exec "$GUNICORN" -c gunicorn.conf.py "$MODULE()"
Executable
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# Development launcher. Pick the service and port:
#
# ./run.sh # main app on 8000
# ./run.sh main 8000 # main app on 8000
# ./run.sh sensors 8001 # sensors app on 8001 (localhost only)
#
# The main app calls the sensors API server-to-server. Override its address
# when the sensors app is not on localhost:8001:
#
# SENSORS_API_URL=http://192.168.0.50:9001 ./run.sh main 8000
#
# The sensors API is open to every user on the local machine. To expose it
# on the network explicitly set HOST=0.0.0.0.
set -euo pipefail
cd "$(dirname "$0")"
APP="${1:-${APP:-main}}"
PORT="${2:-${PORT:-}}"
case "$APP" in
main)
MODULE="app:create_app"
DEFAULT_PORT=8000
DEFAULT_HOST="0.0.0.0"
;;
sensors)
MODULE="sensors_app:create_app"
DEFAULT_PORT=8001
DEFAULT_HOST="127.0.0.1"
;;
*)
echo "Usage: $0 [main|sensors] [port]" >&2
exit 2
;;
esac
PORT="${PORT:-$DEFAULT_PORT}"
HOST="${HOST:-$DEFAULT_HOST}"
if [ -x ".venv/bin/python" ]; then
PYTHON=".venv/bin/python"
else
PYTHON="python3"
fi
echo "Starting $APP app on http://$HOST:$PORT" >&2
exec "$PYTHON" -m flask --app "$MODULE" run --host "$HOST" --port "$PORT" --debug
+1 -1
View File
@@ -3,7 +3,7 @@
import random import random
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from app import create_app from sensors_app import create_app
from models import Sensor, SensorReadout, db from models import Sensor, SensorReadout, db
+183
View File
@@ -0,0 +1,183 @@
from datetime import datetime, timedelta, timezone
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
from flask import Flask, jsonify, request
from config import sensors_database_uri
from models import Sensor, SensorReadout, db
def create_app():
app = Flask(__name__)
app.config["SQLALCHEMY_DATABASE_URI"] = sensors_database_uri()
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
db.init_app(app)
# --- API: device endpoints (intentionally public) ---
# The sensor firmware has no browser session, but it needs the sensor
# list to resolve a name to an id before pushing readouts.
@app.route("/api/sensors")
def api_sensors():
sensors = Sensor.query.order_by(Sensor.name).all()
return jsonify([{"id": s.id, "name": s.name} for s in sensors])
@app.route("/api/sensors", methods=["POST"])
def api_create_sensor():
data = request.get_json(silent=True)
if not data or "name" not in data:
return jsonify({"error": "name is required"}), 400
name = data["name"].strip()
if not name:
return jsonify({"error": "name cannot be empty"}), 400
if Sensor.query.filter_by(name=name).first():
return jsonify({"error": "sensor already exists"}), 409
sensor = Sensor(name=name)
db.session.add(sensor)
db.session.commit()
return jsonify({"id": sensor.id, "name": sensor.name}), 201
@app.route("/api/sensors/<int:sensor_id>/readouts", methods=["POST"])
def api_create_readout(sensor_id):
sensor = db.session.get(Sensor, sensor_id)
if not sensor:
return jsonify({"error": "sensor not found"}), 404
data = request.get_json(silent=True)
if not data:
return jsonify({"error": "request body is required"}), 400
missing = [f for f in ("type", "value") if f not in data]
if missing:
return jsonify({"error": f"missing fields: {', '.join(missing)}"}), 400
try:
value = float(data["value"])
except (TypeError, ValueError):
return jsonify({"error": "value must be a number"}), 400
readout = SensorReadout(
sensor_id=sensor_id,
type=str(data["type"]).strip(),
value=value,
)
db.session.add(readout)
db.session.commit()
return jsonify(
{
"id": readout.id,
"sensor_id": readout.sensor_id,
"type": readout.type,
"value": readout.value,
"timestamp": readout.timestamp.isoformat() + "Z",
}
), 201
# --- API: dashboard reads (public on the local machine) ---
@app.route("/api/sensors/<int:sensor_id>/types")
def api_sensor_types(sensor_id):
types = (
db.session.query(SensorReadout.type)
.filter_by(sensor_id=sensor_id)
.distinct()
.order_by(SensorReadout.type)
.all()
)
return jsonify([t[0] for t in types])
@app.route("/api/sensors/<int:sensor_id>/readouts")
def api_sensor_readouts(sensor_id):
type_ = request.args.get("type")
query = SensorReadout.query.filter_by(sensor_id=sensor_id)
if type_:
query = query.filter_by(type=type_)
readouts = query.order_by(SensorReadout.timestamp.desc()).limit(300).all()
return jsonify(
[
{"value": r.value, "timestamp": r.timestamp.isoformat() + "Z"}
for r in readouts
]
)
@app.route("/api/sensors/<int:sensor_id>/readouts/hourly")
def api_sensor_readouts_hourly(sensor_id):
type_ = request.args.get("type")
range_ = request.args.get("range", "all")
# The frontend sends the browser timezone (e.g. Europe/Warsaw), so
# "today" means "since midnight local time", not UTC.
tz_name = request.args.get("tz", "Europe/Warsaw") or "Europe/Warsaw"
try:
local_tz = ZoneInfo(tz_name)
except (ZoneInfoNotFoundError, ValueError):
return jsonify({"error": f"unknown timezone: {tz_name}"}), 400
# Timestamps are stored as naive UTC, so the cutoff must be naive UTC too.
now_utc = datetime.now(timezone.utc)
now_local = now_utc.astimezone(local_tz)
if range_ == "today":
midnight_local = now_local.replace(hour=0, minute=0, second=0, microsecond=0)
cutoff = midnight_local.astimezone(timezone.utc).replace(tzinfo=None)
elif range_ == "24h":
cutoff = (now_utc - timedelta(hours=24)).replace(tzinfo=None)
elif range_ == "week":
cutoff = (now_utc - timedelta(days=7)).replace(tzinfo=None)
elif range_ == "month":
cutoff = (now_utc - timedelta(days=30)).replace(tzinfo=None)
elif range_ == "all":
cutoff = None
else:
return jsonify({"error": "range must be one of: today, 24h, week, month, all"}), 400
query = db.session.query(SensorReadout.timestamp, SensorReadout.value).filter(
SensorReadout.sensor_id == sensor_id
)
if cutoff is not None:
query = query.filter(SensorReadout.timestamp >= cutoff)
if type_:
query = query.filter(SensorReadout.type == type_)
rows = query.order_by(SensorReadout.timestamp).all()
# today/24h return every measurement by default; the "hourly average"
# checkbox forces hourly averages.
averaged = range_ not in ("today", "24h") or request.args.get(
"avg", "0"
).lower() in ("1", "true", "yes")
if not averaged:
return jsonify(
[
{
"timestamp": ts.strftime("%Y-%m-%dT%H:%M:%S"),
"value": value,
}
for ts, value in rows
]
)
# Other ranges: hourly average over the full local hour (correct
# across DST changes), timestamp is the start of the hour in UTC
# (the frontend appends "Z").
buckets: dict[datetime, list[float]] = {}
for ts, value in rows:
ts_utc = ts.replace(tzinfo=timezone.utc)
hour_local = ts_utc.astimezone(local_tz).replace(
minute=0, second=0, microsecond=0
)
hour_utc = hour_local.astimezone(timezone.utc).replace(tzinfo=None)
buckets.setdefault(hour_utc, []).append(value)
result = [
{
"timestamp": hour.strftime("%Y-%m-%dT%H:00:00"),
"value": round(sum(values) / len(values), 2),
}
for hour, values in sorted(buckets.items())
]
return jsonify(result)
return app
if __name__ == "__main__":
app = create_app()
app.run(debug=True, host='127.0.0.1', port=8001)
+68
View File
@@ -0,0 +1,68 @@
import json
from urllib import error, parse, request
class SensorsAPIError(Exception):
"""Raised when the sensors API is unreachable or returns invalid data."""
class SensorsClient:
def __init__(self, base_url: str, timeout: float = 10.0):
self.base_url = base_url.rstrip("/")
self.timeout = timeout
def _request(self, method, path, params=None, payload=None):
url = f"{self.base_url}{path}"
if params:
query = parse.urlencode({k: v for k, v in params.items() if v is not None})
if query:
url += f"?{query}"
data = None
headers = {"Accept": "application/json"}
if payload is not None:
data = json.dumps(payload).encode("utf-8")
headers["Content-Type"] = "application/json"
req = request.Request(url, data=data, headers=headers, method=method)
try:
with request.urlopen(req, timeout=self.timeout) as response:
body = response.read()
return response.status, self._decode(body)
except error.HTTPError as exc:
return exc.code, self._decode(exc.read())
except (error.URLError, TimeoutError, OSError) as exc:
raise SensorsAPIError(f"sensors API unavailable: {exc}") from exc
@staticmethod
def _decode(body: bytes):
if not body:
return None
try:
return json.loads(body)
except json.JSONDecodeError as exc:
raise SensorsAPIError("sensors API returned invalid JSON") from exc
# --- Sensor data ---
def list_sensors(self):
return self._request("GET", "/api/sensors")
def create_sensor(self, payload):
return self._request("POST", "/api/sensors", payload=payload)
def create_readout(self, sensor_id, payload):
return self._request(
"POST", f"/api/sensors/{sensor_id}/readouts", payload=payload
)
def sensor_types(self, sensor_id):
return self._request("GET", f"/api/sensors/{sensor_id}/types")
def sensor_readouts(self, sensor_id, params=None):
return self._request("GET", f"/api/sensors/{sensor_id}/readouts", params=params)
def sensor_readouts_hourly(self, sensor_id, params=None):
return self._request(
"GET", f"/api/sensors/{sensor_id}/readouts/hourly", params=params
)
+8 -7
View File
@@ -18,7 +18,8 @@
<h1>Sensor Readouts &mdash; API Documentation</h1> <h1>Sensor Readouts &mdash; API Documentation</h1>
<h2>Base URL</h2> <h2>Base URL</h2>
<p><code>http://localhost:5000/api</code></p> <p><code>{{ main_api_url }}</code></p>
<p>The main app proxies all requests to the sensors API service.</p>
<!-- GET /api/sensors --> <!-- GET /api/sensors -->
<h2>GET /api/sensors</h2> <h2>GET /api/sensors</h2>
@@ -26,7 +27,7 @@
<span class="method get">GET</span><span class="path">/api/sensors</span> <span class="method get">GET</span><span class="path">/api/sensors</span>
<p class="desc">Return a list of all sensors ordered by name.</p> <p class="desc">Return a list of all sensors ordered by name.</p>
<p class="section-label">curl</p> <p class="section-label">curl</p>
<pre><code>curl http://localhost:5000/api/sensors</code></pre> <pre><code>curl {{ main_api_url }}/sensors</code></pre>
<p class="section-label">Response 200</p> <p class="section-label">Response 200</p>
<pre><code>[ <pre><code>[
{ "id": 1, "name": "outdoor-station" }, { "id": 1, "name": "outdoor-station" },
@@ -39,7 +40,7 @@
<span class="method post">POST</span><span class="path">/api/sensors</span> <span class="method post">POST</span><span class="path">/api/sensors</span>
<p class="desc">Create a new sensor.</p> <p class="desc">Create a new sensor.</p>
<p class="section-label">curl</p> <p class="section-label">curl</p>
<pre><code>curl -X POST http://localhost:5000/api/sensors \ <pre><code>curl -X POST {{ main_api_url }}/sensors \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{"name": "new-sensor"}'</code></pre> -d '{"name": "new-sensor"}'</code></pre>
<p class="section-label">Response 201</p> <p class="section-label">Response 201</p>
@@ -58,7 +59,7 @@
<span class="method get">GET</span><span class="path">/api/sensors/:id/types</span> <span class="method get">GET</span><span class="path">/api/sensors/:id/types</span>
<p class="desc">Return distinct readout types available for a given sensor.</p> <p class="desc">Return distinct readout types available for a given sensor.</p>
<p class="section-label">curl</p> <p class="section-label">curl</p>
<pre><code>curl http://localhost:5000/api/sensors/1/types</code></pre> <pre><code>curl {{ main_api_url }}/sensors/1/types</code></pre>
<p class="section-label">Response 200</p> <p class="section-label">Response 200</p>
<pre><code>["humidity", "pressure", "temperature"]</code></pre> <pre><code>["humidity", "pressure", "temperature"]</code></pre>
<p><strong>Errors:</strong></p> <p><strong>Errors:</strong></p>
@@ -80,10 +81,10 @@
</table> </table>
<p class="section-label">curl</p> <p class="section-label">curl</p>
<pre><code># all types <pre><code># all types
curl http://localhost:5000/api/sensors/1/readouts curl {{ main_api_url }}/sensors/1/readouts
# filter by type # filter by type
curl "http://localhost:5000/api/sensors/1/readouts?type=temperature"</code></pre> curl "{{ main_api_url }}/sensors/1/readouts?type=temperature"</code></pre>
<p class="section-label">Response 200</p> <p class="section-label">Response 200</p>
<pre><code>[ <pre><code>[
{ "value": 24.31, "timestamp": "2026-09-11T10:30:00" }, { "value": 24.31, "timestamp": "2026-09-11T10:30:00" },
@@ -96,7 +97,7 @@ curl "http://localhost:5000/api/sensors/1/readouts?type=temperature"</code></pre
<span class="method post">POST</span><span class="path">/api/sensors/:id/readouts</span> <span class="method post">POST</span><span class="path">/api/sensors/:id/readouts</span>
<p class="desc">Add a new readout to an existing sensor. Timestamp is generated server-side.</p> <p class="desc">Add a new readout to an existing sensor. Timestamp is generated server-side.</p>
<p class="section-label">curl</p> <p class="section-label">curl</p>
<pre><code>curl -X POST http://localhost:5000/api/sensors/1/readouts \ <pre><code>curl -X POST {{ main_api_url }}/sensors/1/readouts \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{"type": "temperature", "value": 22.5}'</code></pre> -d '{"type": "temperature", "value": 22.5}'</code></pre>
<p class="section-label">Response 201</p> <p class="section-label">Response 201</p>
+200
View File
@@ -0,0 +1,200 @@
import os
import tempfile
import threading
import unittest
from pathlib import Path
from sqlalchemy import inspect
from werkzeug.serving import make_server
ENV_KEYS = (
"SENSORS_DATABASE_URL",
"USERS_DATABASE_URL",
"SECRET_KEY",
"SENSORS_API_URL",
)
def _save_env(keys):
return {key: os.environ.get(key) for key in keys}
def _restore_env(saved):
for key, value in saved.items():
if value is None:
os.environ.pop(key, None)
else:
os.environ[key] = value
class SensorApiTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.tmpdir = tempfile.TemporaryDirectory()
cls._env = _save_env(ENV_KEYS)
os.environ["SENSORS_DATABASE_URL"] = (
f"sqlite:///{Path(cls.tmpdir.name) / 'sensors.db'}"
)
from models import db
from sensors_app import create_app
cls.app = create_app()
with cls.app.app_context():
db.create_all()
cls.client = cls.app.test_client()
@classmethod
def tearDownClass(cls):
from models import db
with cls.app.app_context():
db.session.remove()
db.drop_all()
cls.tmpdir.cleanup()
_restore_env(cls._env)
def test_sensors_db_has_no_user_table(self):
from models import db
with self.app.app_context():
tables = set(inspect(db.engine).get_table_names())
self.assertEqual(tables, {"sensor", "sensor_readout"})
def test_all_api_endpoints_are_public(self):
created = self.client.post("/api/sensors", json={"name": "api-test-sensor"})
self.assertEqual(created.status_code, 201)
sensor_id = created.get_json()["id"]
readout = self.client.post(
f"/api/sensors/{sensor_id}/readouts",
json={"type": "temperature", "value": 20.5},
)
self.assertEqual(readout.status_code, 201)
self.assertEqual(self.client.get("/api/sensors").status_code, 200)
types = self.client.get(f"/api/sensors/{sensor_id}/types")
self.assertEqual(types.status_code, 200)
self.assertEqual(types.get_json(), ["temperature"])
readouts = self.client.get(f"/api/sensors/{sensor_id}/readouts")
self.assertEqual(readouts.status_code, 200)
self.assertEqual(readouts.get_json()[0]["value"], 20.5)
hourly = self.client.get(
f"/api/sensors/{sensor_id}/readouts/hourly?type=temperature&range=all"
)
self.assertEqual(hourly.status_code, 200)
self.assertEqual(len(hourly.get_json()), 1)
class MainAppTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.tmpdir = tempfile.TemporaryDirectory()
cls._env = _save_env(ENV_KEYS)
base = Path(cls.tmpdir.name)
os.environ["SENSORS_DATABASE_URL"] = f"sqlite:///{base / 'sensors.db'}"
os.environ["USERS_DATABASE_URL"] = f"sqlite:///{base / 'main_app.db'}"
os.environ["SECRET_KEY"] = "test-secret"
from models import db
from sensors_app import create_app as create_sensors_app
from user_models import User, user_db
cls.sensors_app = create_sensors_app()
with cls.sensors_app.app_context():
db.create_all()
cls.server = make_server("127.0.0.1", 0, cls.sensors_app)
cls.thread = threading.Thread(target=cls.server.serve_forever, daemon=True)
cls.thread.start()
os.environ["SENSORS_API_URL"] = f"http://127.0.0.1:{cls.server.server_port}"
from app import create_app as create_main_app
cls.main_app = create_main_app()
with cls.main_app.app_context():
user_db.create_all()
user = User(username="tester")
user.set_password("secret")
user_db.session.add(user)
user_db.session.commit()
def setUp(self):
self.client = self.main_app.test_client()
@classmethod
def tearDownClass(cls):
from models import db
from user_models import user_db
cls.server.shutdown()
cls.thread.join()
with cls.sensors_app.app_context():
db.session.remove()
db.drop_all()
with cls.main_app.app_context():
user_db.session.remove()
user_db.drop_all()
cls.tmpdir.cleanup()
_restore_env(cls._env)
def login(self):
return self.client.post(
"/login", data={"username": "tester", "password": "secret"}
)
def create_sensor_and_readout(self, name):
created = self.client.post("/api/sensors", json={"name": name})
self.assertEqual(created.status_code, 201)
sensor_id = created.get_json()["id"]
readout = self.client.post(
f"/api/sensors/{sensor_id}/readouts",
json={"type": "temperature", "value": 21.5},
)
self.assertEqual(readout.status_code, 201)
return sensor_id
def test_views_and_login_are_owned_by_main_app(self):
response = self.client.get("/sensors")
self.assertEqual(response.status_code, 302)
self.assertIn("/login", response.headers["Location"])
self.assertEqual(self.client.get("/login").status_code, 200)
self.assertEqual(self.login().status_code, 302)
self.assertEqual(self.client.get("/home").status_code, 200)
self.assertEqual(self.client.get("/sensors").status_code, 200)
self.assertEqual(self.client.get("/api/docs").status_code, 200)
def test_public_device_api_is_proxied_to_sensors(self):
sensor_id = self.create_sensor_and_readout("proxy-test")
listed = self.client.get("/api/sensors").get_json()
self.assertIn({"id": sensor_id, "name": "proxy-test"}, listed)
def test_dashboard_reads_require_login_and_are_proxied(self):
sensor_id = self.create_sensor_and_readout("dashboard-test")
self.assertEqual(self.client.get(f"/api/sensors/{sensor_id}/types").status_code, 401)
self.login()
types = self.client.get(f"/api/sensors/{sensor_id}/types")
self.assertEqual(types.status_code, 200)
self.assertEqual(types.get_json(), ["temperature"])
readouts = self.client.get(f"/api/sensors/{sensor_id}/readouts?type=temperature")
self.assertEqual(readouts.status_code, 200)
self.assertEqual(readouts.get_json()[0]["value"], 21.5)
hourly = self.client.get(
f"/api/sensors/{sensor_id}/readouts/hourly?type=temperature&range=all"
)
self.assertEqual(hourly.status_code, 200)
self.assertEqual(len(hourly.get_json()), 1)
if __name__ == "__main__":
unittest.main()
+25
View File
@@ -0,0 +1,25 @@
from datetime import datetime, timezone
from flask_login import UserMixin
from flask_sqlalchemy import SQLAlchemy
from werkzeug.security import check_password_hash, generate_password_hash
user_db = SQLAlchemy()
class User(UserMixin, user_db.Model):
__tablename__ = "user"
id = user_db.Column(user_db.Integer, primary_key=True)
username = user_db.Column(user_db.String(80), nullable=False, unique=True)
password_hash = user_db.Column(user_db.String(255), nullable=False)
is_admin = user_db.Column(user_db.Boolean, nullable=False, default=False)
created_at = user_db.Column(
user_db.DateTime, default=lambda: datetime.now(timezone.utc)
)
def set_password(self, password: str) -> None:
self.password_hash = generate_password_hash(password)
def check_password(self, password: str) -> bool:
return check_password_hash(self.password_hash, password)
+1 -1
View File
@@ -1 +1 @@
ssh -R 5432:localhost:8000 vps ssh -R 5432:localhost:8001 vps
+18
View File
@@ -0,0 +1,18 @@
[Unit]
Description=Yggdrasil Sensors App
After=network.target yggdrasil.service
[Service]
User=mateusz
Group=mateusz
WorkingDirectory=/home/mateusz/prog/yggdrasil/server
# The sensors API is open to local users and listens on localhost only.
# Data lives in sensors.db; users live in the main app's main_app.db.
#Environment=SENSORS_DB_PATH=/home/mateusz/prog/yggdrasil/server/sensors.db
#Environment=HOST=0.0.0.0
ExecStart=/home/mateusz/prog/yggdrasil/server/run-prod.sh sensors 8001
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
+5 -2
View File
@@ -1,12 +1,15 @@
[Unit] [Unit]
Description=Yggdrasil Sensor API Description=Yggdrasil Main App
After=network.target After=network.target
[Service] [Service]
User=mateusz User=mateusz
Group=mateusz Group=mateusz
WorkingDirectory=/home/mateusz/prog/yggdrasil/server WorkingDirectory=/home/mateusz/prog/yggdrasil/server
ExecStart=/home/mateusz/prog/yggdrasil/server/.venv/bin/gunicorn -c gunicorn.conf.py "app:create_app()" # Override when the sensors API is not on localhost:8001:
#Environment=SENSORS_API_URL=http://localhost:8001
#Environment=USERS_DB_PATH=/home/mateusz/prog/yggdrasil/server/main_app.db
ExecStart=/home/mateusz/prog/yggdrasil/server/run-prod.sh main 8000
Restart=on-failure Restart=on-failure
RestartSec=5 RestartSec=5