46 lines
1.3 KiB
Markdown
46 lines
1.3 KiB
Markdown
# Preprocess Service
|
|
|
|
Reads unprocessed `raw_packets` from PostgreSQL and builds aircraft, flights, tracks, and track_points.
|
|
|
|
## What it does
|
|
|
|
1. Waits for PostgreSQL to be ready
|
|
2. Reads `fr24.processing_state` to find the last processed `raw_packet_id`
|
|
3. Fetches the next batch of unprocessed packets
|
|
4. For each packet: upserts `aircraft`, creates/reuses a `flight`, appends a `track_point`
|
|
5. Advances the cursor in `processing_state`
|
|
6. Touches `/tmp/preprocess-ready` for Docker healthcheck
|
|
|
|
Data is fake/test for this stage — real ADS-B decoding comes later.
|
|
|
|
## Dependencies
|
|
|
|
- `psycopg2-binary`
|
|
|
|
## Environment variables
|
|
|
|
| Variable | Default | Description |
|
|
|---|---|---|
|
|
| `POSTGRES_HOST` | required | DB host |
|
|
| `POSTGRES_PORT` | `5432` | DB port |
|
|
| `POSTGRES_DB` | required | DB name |
|
|
| `POSTGRES_USER` | required | DB user |
|
|
| `POSTGRES_PASSWORD` | required | DB password |
|
|
| `POLL_INTERVAL_SECONDS` | `5.0` | How often to poll for new packets |
|
|
| `BATCH_SIZE` | `20` | Packets per processing batch |
|
|
|
|
## Run locally
|
|
|
|
```bash
|
|
pip install -r requirements.txt
|
|
export POSTGRES_HOST=localhost POSTGRES_DB=fr24 POSTGRES_USER=fr24 POSTGRES_PASSWORD=change-me
|
|
python main.py
|
|
```
|
|
|
|
## Build & run via Docker
|
|
|
|
```bash
|
|
docker build -t fr24-preprocess .
|
|
docker run --env-file ../../compose/.env fr24-preprocess
|
|
```
|