todos-contra-el-fuego-web/UPGRADE.md

Meteor upgrade — 1.6.1.1 → 2.5 (Mongo 3.2) → 3.1 (Mongo 7, on meteor3-wip)

Incremental, verified upgrade of todos-contra-el-fuego-web. Branch: meteor3-upgrade (base tcef-master). Local commits only — no push until authorized.

⚠️ Hard blocker for the final 3.x jump: MongoDB 3.2

Production runs against the shared replica set rsmain on MongoDB 3.2.11. Meteor 3 ships the modern mongodb Node driver, which requires a server ≥ 4.2/4.4. Meteor 2.x still works against Mongo 3.2.

Therefore this branch lands on Meteor 2.5 — empirically the highest release whose bundled mongodb driver still connects to Mongo 3.2 (2.6 bumps the driver to 4.x and fails; see "Escala ceiling" below) — with everything green. The meteor update --release 3.x step (and even 2.6+) is deliberately not taken until the Comunes infra team upgrades rsmain (shared infra — coordinated in a separate phase, not unilaterally). Server code should still be migrated to async/await before the final 3.x jump so it is mechanical (tracked as debt).

⚠️ Deploy-ordering dependency: notifications cutover

The old push/email notification code was removed from the web (see the notifications commit) because it now lives in the tcef-notifications microservice. Production must not run this build until tcef-notifications is emitting in prod — otherwise users stop receiving notifications. The web deploy and the notifications cutover must be coordinated.

Safety net: REST smoke test

smoke/ is a standalone Node harness that calls every REST endpoint the Flutter app consumes, against a seeded local dev server, and diffs the responses against committed snapshots (smoke/snapshots/). See smoke/README.md.

Run after every escala — it must stay byte-identical:

docker run -d --name tcef-mongo32 -p 27018:27017 mongo:3.2   # once
export PATH="$HOME/.meteor:$PATH" MONGO_URL="mongodb://localhost:27018/fuegos"
meteor --settings settings-development.json --port 3100 &     # dev server
./smoke/smoke.sh                                              # seed + compare

Escalas

Baseline — Meteor 1.6.1.1 (starting point)

• Boots against Mongo 3.2 with settings-development.json.
• Dev-enabling changes (do not affect production behavior):
  • IPGeocoder.js: degrade gracefully when the MaxMind GeoLite2 DB is absent (provisioned by cron in prod) instead of crashing at boot.
  • settings-development.json: added dev-only private.internalApiToken (dev-smoke-token) — the REST API only registers its routes when this is set.
• Notifications code removed (migrated to tcef-notifications): deleted notificationsObserver.js, notificationsProcess.js, the "Process pending notif" SyncedCron job, and the node-gcm dependency. subsUnion.js kept (it feeds subs-public-union, consumed by the app).
• REST smoke baseline captured here. ✅ green.

Escala 1 — 1.6.1.1 → 1.8.3 ✅ green

meteor update --release 1.8.3
meteor npm install --save @babel/runtime@^7.26.0

Core packages bumped (highlights): meteor 1.8.6→1.9.3, ecmascript 0.10.7→0.13.2, modules 0.11.6→0.14.0, mongo 1.4.7→1.7.0, npm-mongo 2.2.34→3.2.0 (mongodb driver 3.x — still fine against Mongo 3.2 server), webapp 1.5.0→1.7.5, standard-minifier-js →2.5.2. underscore auto-added (1.7 dropped it from meteor-base), fetch + modern-browsers added.

Breaking change — Babel beta → stable. Boot failed with Cannot find module '@babel/runtime/helpers/objectSpread2': the pinned @babel/runtime@7.0.0-beta.44 predates helpers that 1.8's ecmascript emits. Fixed by upgrading to stable 7.x (@babel/runtime@^7.29.7). Note: @latest resolves to 8.x, which is too new for Meteor's 7-style helper layout — pin to ^7.

REST smoke test: byte-identical to baseline. ✅

Escala 2 — 1.8.3 → 1.11 (version resolution only)

meteor update --release 1.11

Resolution reached 1.11 (ecmascript 0.14.3, mongo 1.10.0, npm-mongo 3.8.0 — still Mongo-3.2-compatible, accounts-* 1.x, email 2.0). But this release surfaced the two big blockers below. The app was ultimately driven to 2.3 (see next section); 1.11 was a transit point, not a landing.

Escala 3 — → Meteor 2.3 ✅ green (builds + runs against Mongo 3.2, smoke byte-identical)

.meteor/release = METEOR@2.3. npm-mongo 3.9.0 (mongodb driver 3.x) — the last driver line that still speaks to a Mongo 3.2 server. Core bumps: accounts-* 2.0, ecmascript 0.15.2, mongo 1.12.0, HMR added.

This escala was dominated by dead Atmosphere packages (the plan's #1 risk). Root causes and fixes, in the order they were hit:

1. arkham:comments-ui — no Meteor 2.x build. The vendored local package used Npm.depends, and rebuilding a local package's npm deps crashes meteor-tool ≥1.11 on this host (node_contextify.cc … Assertion args[1]->IsString() → SIGABRT; reproducible with any Npm.depends, even is-number). Switching to the Atmosphere isopack avoided the local rebuild but the package is abandoned: its newest 2.x-solvable version (0.2.15) pins accounts-password@1.0.1, conflicting with Meteor 2.0's accounts-password 2.0. Decision (user-approved): reimplement comments as a small React feature (imports/api/Comments/*, imports/ui/components/Comments/CommentsBox.js), dropping the package entirely. Fire-page comments only (the sole usage), with likes/dislikes, image/YouTube embed, owner edit/remove, and the "email other commenters" side-effect preserved. Comment text is now rendered as safe plain text (+ media embed) instead of markdown-to-HTML — a small scope reduction and a security improvement (no raw-HTML injection of user content). The old local fork is preserved in its nested git repo (commit 4761da9).

2. coffeescript@1.0.17 build plugin — crashes the build. The same node_contextify assertion fired at build start on this host whenever the coffeescript compiler plugin loaded. It was pulled transitively by the dead test stack (meteortesting:mocha, practicalmeteor:chai, xolvio:cleaner) and by the old auto-downgraded nimble:restivus@0.6.6 (which needs iron:router). Removing those removed coffeescript and unblocked the build. A trivial meteor create app builds fine on the same tool, confirming the crash is package-specific, not a broken tool.

3. nimble:restivus — the REST API package — pins accounts-password@1.3.3 (incompatible with 2.x) and is CoffeeScript (needs the crashing plugin). Vendored as packages/nimble-restivus: the .coffee sources were precompiled to plain JS (lib/restivus-all.js, single translation unit so the Auth/Route/Restivus classes share scope; Restivus left un-var'd so Meteor's api.export picks it up), and package.js loosens accounts-password to 2.x and drops the coffeescript dependency. This also let restivus resolve to 0.8.12 (json-routes based), which removed the whole iron:router stack. REST behavior unchanged (smoke byte-identical).

4. maximum:server-transform (unused) removal broke Meteor.publishTransformed in FalsePositives/server/publications.js; no transform was actually configured, so replaced with plain Meteor.publish.

5. fourseven:scss 4.5.4 → 4.14.1 — node-sass@4.5.3 won't build on node 12 (Meteor ≥1.9); 4.14.1 uses a node-12-compatible node-sass with prebuilt binaries. (Meteor 1.8 built it on node 8.) 4.15.0 needs ecmascript ≥ 0.15.1 → too new for 1.11.

6. less, markdown removed — no .less/.md source files; their build plugins were dead weight.

⚠️ Local build-host caveat

Meteor-tool's bundled node (12.x/14.x) SIGABRTs (node_contextify assertion) on this machine (kernel 6.12) when certain old build plugins load — this is an environment interaction, not a code defect (the deploy host built these fine historically). It was fully worked around by removing the dead build plugins above. If a future escala hits it again, build in a container matching the deploy target (Debian) rather than on this host.

Escalas 2.4 & 2.5 — ✅ green

meteor update --release 2.4, then --release 2.5. Both trivial; npm-mongo stays at 3.9.1 (mongodb driver 3.x). REST smoke byte-identical against mongo:3.2 on both.

Escala ceiling vs Mongo 3.2 — landing on Meteor 2.5 (empirically verified)

Meteor bumps the bundled mongodb Node driver to 4.3.1 at Meteor 2.6, which requires server wire version ≥ 6 (MongoDB ≥ 3.6). Production's rsmain is Mongo 3.2 (wire version 4). Verified directly by booting 2.6 against a real mongo:3.2:

MongoCompatibilityError: Server at localhost:27018 reports maximum wire
version 4, but this version of the Node.js Driver requires at least 6
(MongoDB 3.6)
=> Exited with code: 1

So Meteor 2.5 is the highest release that runs against the production Mongo 3.2 (npm-mongo 3.9.1). That is where this branch lands: everything green, REST smoke byte-identical. Going past 2.5 (to 2.16 or 3.x) is gated on upgrading the shared rsmain replica set to Mongo ≥ 4.4 first — the same DB blocker as 3.x.

✅ Meteor 3.1 + MongoDB 7 — REST smoke GREEN (branch meteor3-wip)

The 3.x jump works end-to-end for the REST/Flutter contract. On branch meteor3-wip, the web runs on Meteor 3.1 (Node 22, async/await, no Fibers) against a dockerized MongoDB 7 (npm-mongo 6.10 / mongodb driver 6), and the REST smoke test is byte-identical to the 1.6.1.1 baseline (all 12 endpoints). This is the "mongo superior" decision realized: give fuegos its own Mongo 7 instead of the shared rsmain 3.2 — which is what unblocks Meteor 3's driver.

meteor3-upgrade stays at the deployable Meteor 2.5 (Mongo 3.2). The 3.x work lives on meteor3-wip until (a) production Mongo is migrated to 7 and (b) the web-UI async migration below is finished.

Reproduce the 3.1 dev stack

docker run -d --name tcef-mongo7 -p 27019:27019 mongo:7 --replSet rs0 --port 27019 --bind_ip_all
docker exec tcef-mongo7 mongosh --port 27019 --quiet --eval 'rs.initiate({_id:"rs0",members:[{_id:0,host:"localhost:27019"}]})'
# (migrations run from scratch on an empty DB — all up() bodies are async now,
#  no need to pre-mark db.migrations at version 18)
export MONGO_URL="mongodb://localhost:27019/fuegos?replicaSet=rs0"
export NODE_OPTIONS="--dns-result-order=ipv4first --no-network-family-autoselection"  # reach warehouse.meteor.com on Node 22
meteor --settings settings-development.json --port 3100
MONGO_CONTAINER=tcef-mongo7 MONGO_SHELL=mongosh MONGO_PORT=27019 ./smoke/smoke.sh

Key build/runtime unblocks (Meteor 3.1)

• underscore linker crash (Runtime is not available … underscore): resolved underscore@1.6.2 had a runtime-less web.browser unibuild → pin underscore@1.6.4 (diagnosed by instrumenting the tool's linker.js).
• collection2 v4 is fully lazy with no main module → import its eager entry meteor/aldeed:collection2/static.js from server|client/00-collection2-init.js so attachSchema/autoValues are patched before any collection loads.
• Fibers gone: removed fibers.js; server sync Mongo → *Async throughout (Rest.js + helpers + methods + startup: oauth/subsUnion/migrations/email/facts).
• vendored restivus patched to await async endpoint handlers.
• $near + countAsync(): driver 6 runs count via aggregation where $near is illegal → derive total from fetchAsync().length.
• json-routes 3.0 matches routes in registration order → reordered the mobile/subscriptions/all/... route before .../:subsId in Rest.js.
• fixtures.js / sitemaps.js disabled (sync-only @cleverbeagle/seeder / pre-0.9 gadicohen:sitemaps) — not on the REST path. Debt.

Web-UI async migration (beyond the REST contract)

• ✅ Fires publications → async (fireFromId, fireFromAlertId, fireFromActiveId, fireFromHash): findOne→findOneAsync, count()→countAsync(), firesUnion awaited, handlers async so the try/catch actually catches rejections. Verified over raw DDP (all subs ready, docs flow; fire + falsePositives docs delivered for the seeded fire). falsePositivesMyloc / industriesMyloc / comments.forReference needed no change — they return bare find() cursors (still sync-safe on Meteor 3). subscriptions.* methods were already *Async.
• Note: the module-level new Counter(...) (natestrauser:publish-performant-counts) behind falsePositivesTotal boots fine; the publication itself is unused by the current UI — watch it if re-enabled.
• ✅ Comments methods → async (comments.insert/edit/remove/like/dislike): findOne/insert/update/remove → *Async, helpers (requireOwner, toggle) async, users.findOneAsync for the username. onCommentAdd mail fan-out: users.find().forEach → forEachAsync. Still fire-and-forget from the insert. Comments UI staging verification pending (pre-existing note below).
• ✅ fixtures.js re-enabled — @cleverbeagle/seeder (sync Mongo, no Meteor 3 support) removed from package.json and replaced with a small in-repo async seeder: idempotent, dev-only (staging opts in with TCEF_SEED=1); same accounts as before (admin@admin.com / 5 test users / 5 Documents each). Verified: fixtures: 6 dev users ensured on boot.
• ✅ sitemaps.js re-enabled — gadicohen:sitemaps (pre-0.9 API, dead on Meteor 3) removed from .meteor/packages; /sitemap.xml is now a small WebApp.connectHandlers handler serving the same static page list (the per-fire section of the old handler was behind firesMapEnabled = false, i.e. dead code, and was dropped). Verified serving on dev.
• ✅ Dead client packages purged — the web UI now RENDERS on Meteor 3. First-ever browser verification of this branch (the smoke only covers REST) found a chain of ancient Blaze-era packages whose client code threw at bundle load, killing the whole app (meteorInstall is not defined):
  • alanning:roles 1.2.10 (client crash, and its server Roles.userIsInRole is sync) → package removed; the app barely used it: facts.js now checks the plain user.roles field (admin set cached at startup) and App.js takes roles from the user doc (prop was unused). Upgrading to roles v4 was rejected: needs a prod data migration.
  • selaias:cookie-consent (its Cookies global is gone) → removed; replaced by in-repo React imports/ui/components/CookieConsent reusing the same i18n keys; CookieConsent.init dropped from i18n.js.
  • natestrauser:publish-performant-counts server Counter._publishCursor uses sync cursor.count() → vendored as packages/publish-performant-counts (tcef:publish-performant-counts) with countAsync and an async-aware interval; same public API.
  • Meteor.autorun (removed in Meteor 3) → dropped (App.js debug) / Tracker.autorun (FiresMap).
  • Map/server publications not exercised by the REST smoke converted to async: activefiresmyloc, activefiresunionmyloc, fireAlerts (countAsync/fetchAsync/awaited firesUnion), and oauth.verifyConfiguration (findOneAsync). migrations.js: all 18 historical up() bodies ported to async (fetchAsync+for..of, *Async writes, createIndexAsync, Accounts.createUserAsync; percolate:migrations 2.0.1 awaits async up()). Verified: empty Mongo 7 DB migrates 0→18 with no errors (indexes + industry registries created); pre-marking version=18 is no longer needed. Verified in a real browser: / and /fires render (map, search, layers, cookie banner), zero uncaught exceptions; REST smoke byte-identical.
• ✅ React 16 → 18.3 — react/react-dom bumped to ^18.3.1 (--legacy-peer-deps: the ancient react-* libs below declare 15/16 peers), client entry migrated to createRoot (imports/startup/client/index.js). The legacy UI libs (react-bootstrap 0.31, reactstrap 5-alpha, react-leaflet 1.8, react-router-dom 4, react-i18next 7) stay pinned and WORK on 18, but spam the console with legacy-context/defaultProps deprecation warnings and will die on React 19. Debt: own front-end refresh project. ⚠️ npm gotcha: running meteor npm install/uninstall while the dev server watches node_modules can crash the meteor-tool watcher mid-prune (ENOENT on a watched file). Stop the server (or restart it after) when touching deps.

Still TODO before the web (not just REST) is fully deployable on 3.x

• React 16 → 18 render root + verify ancient react-* libs.
• Re-enable dev fixtures / sitemaps / email default template with async ports.
• SCSS @import→@use, lighten/darken→color.adjust (dart-sass deprecations).
• Production cutover still needs: Mongo data migrated 3.2→7 (mongodump/restore), the notifications service emitting in prod, and the Docker deployment (fase 3).

Dependency debt (tracked, to resolve in the noted escala)

Dep	From	Target	Status
React / react-dom	16.0	18	✅ done — 18.3.1 + createRoot; ancient react-* libs work on 18 via legacy context (die on 19, debt).
Server async/await	Fibers	*Async APIs	✅ done — REST, publications, methods, startup and migrations all on *Async.
bcrypt	1.0.3 (broken binding)	5.x	✅ done — 1.0.3 didn't load on Node 22 (accounts-password silently fell back to pure JS) and wouldn't compile in the Docker build; 5.1.1 ships Node-22 prebuilds.
raven / flowkey:raven	2.4	@sentry/node + @sentry/browser 8.x	✅ done — flowkey:raven (Sentry legacy SDK, dead on 3.x, spammed the boot log with 502s against the dead sentry.comunes.org) replaced by the modern SDKs behind the same ravenLogger.log() facade (6 call sites unchanged). Backend: GlitchTip self-hosted on aaron (Sentry-compatible DSN/API; Sentry proper needs 16 GB RAM, didn't fit), deployed via the Comunes ansible (glitchtip.yml, org "comunes" / project "tcef-web"), fronted at https://sentry.comunes.org (nginx on assange → aaron:8000). DSN set in settings-development.json; end-to-end verified (a test exception sent from the dev server showed up in GlitchTip within seconds).
nodemailer	4	6.10	✅ done — drop-in for our usage: email.js only calls nodemailer.createTransport(MAIL_URL) and reads transport.options.auth.user (in the production-only from() branch); both verified unchanged in v6. Sending goes through ostrio:mailer (MailTime 2.5), which is v6-compatible.
Babel	7 beta	7 stable	✅ done (escala 1)
react-meteor-data	0.2.16	3.0.6	✅ done — 0.2.16 (React-15/16-era) looped withTracker on fire-detail pages under React 18 (never-ready → blank). meteor add react-meteor-data@3.0.6; no call-site changes (HOC API kept). Also dropped tmeasday:check-npm-versions (the constraint pinning 0.2.16). See section above.
i18next	10	current	debt — pin; not on the critical path
Leaflet	1.3.1	current	debt — pin; not on the critical path
arkham:comments-ui	1.4.x	—	✅ replaced with in-repo React feature
nimble:restivus	Atmosphere	vendored	✅ local precompiled package, accounts-password 2.x
maximum:server-transform, meteorhacks:zones, less, markdown, test stack	—	—	✅ removed (dead/unused)
fourseven:scss	4.5.4	4.14.1	✅ node-12 compatible
node-gcm	1.0.2	—	✅ removed (dead API, moved to microservice)
chimp	0.51.1	—	✅ removed — dead e2e test runner (Selenium 2 / cucumber), sole remaining puller of native fibers@1.x, which cannot compile on Node 22 and broke the clean Docker build. Not imported anywhere.

Comments React feature — verification note

The server builds and the REST smoke test is green, and the client bundle compiles, but the comments UI itself is not exercised by the smoke test (it's not part of the Flutter REST contract). It needs manual staging verification: on a fire archive page — logged-in post/edit/remove, like/dislike, and image/YouTube embed.

✅ Fixed: fire-detail pages now render (react-meteor-data 0.2.16 → 3.0.6)

Browser-verifying the Comments feature surfaced a pre-existing client bug: the fire-detail pages (/fire/{active,archive,alert}/:id, component imports/ui/pages/Fires/Fires.js) stayed blank — the page's withTracker subscription (fireFromActiveId / fireFromId) never reached ready(), so loading stayed true and nothing rendered, pegging a CPU core in a re-subscribe loop.

Diagnosis (isolated, not guessed):

• The publications were fine. Subscribing to fireFromActiveId / fireFromId standalone from the browser console readied immediately and delivered the doc — not a server/publication bug.
• The bug was react-meteor-data@0.2.16 — a React-15/16-era version (componentWillMount + legacy context, the source of the deprecation warnings flooding the console) that misbehaves under React 18 for a component gating its whole render on subscription.ready().

Fix: meteor add react-meteor-data@3.0.6 (the React-18-compatible line for Meteor 3.1; 4.x targets Meteor 3.2+). No call-site changes were needed — 3.x keeps the withTracker HOC API, and all 19 call sites work unchanged. The old constraint that pinned 0.2.16 was tmeasday:check-npm-versions, pulled in by 0.2.16 itself; the upgrade dropped it, which also silenced the "npm peer requirements not installed (react@15-16)" boot/console warning.

Verified in a real browser at /fire/archive/c0000000000000000000000c: the page renders fully (title, Leaflet + NASA/satellite map, the "not a wildfire" FalsePositives widget, and the Comments box — all withTracker-driven), zero console errors. REST smoke byte-identical (12/12).