process.cwd() for this process is already /app/programs/server (same
basis WORKER_PATH already relied on), so prefixing 'programs/server/'
again produced .../programs/server/programs/server/npm/node_modules —
still Cannot find module '@turf/circle', just with a longer wrong path.
The previous side-effect-import fix didn't work: the app's own npm deps
(installed by `meteor build`) live in programs/server/npm/node_modules,
a SIBLING of programs/server/assets, not an ancestor of it. Node's
require() only walks up parent directories looking for node_modules,
so require('@turf/circle') from the worker asset script could never
find it regardless of what gets imported elsewhere in the app — it
kept failing with "Cannot find module '@turf/circle'" even after that
fix was deployed.
calcUnionAsync.js (Meteor-compiled code, which already knows how to
locate the app's npm deps) now resolves the three turf package paths
itself and hands them to the worker via workerData; the worker
requires them by absolute path instead of bare specifier.
storeUnion checked `typeof union === 'object'` as its "did this work"
guard, but typeof null is also 'object' — so a worker call that failed
(caught upstream, leaving union as its initial null) looked identical
to a legitimate "zero valid subscriptions" result, and got silently
stored as the string "null" instead of being treated as a failure.
This is exactly what corrupted subs-public-union/subs-private-union on
staging just now, while chasing the missing-turf-deps bug.
storeUnion is now only called once calcUnionAsync has actually
resolved; process() and incrementalAdd() catch failures themselves and
skip storing anything, leaving the previous (still valid) union alone.
Meteor's build only includes npm packages it sees imported/required
somewhere in its own traced module graph. private/workers/unionWorker.js
is a plain asset (loaded at runtime via new Worker(), never
import/require'd from Meteor-compiled code), so @turf/circle,
@turf/union and @turf/truncate got silently dropped from
programs/server/node_modules — the worker crashed on every call with
"Cannot find module '@turf/circle'", which incrementalAdd/process
swallowed and stored as a "null" union (typeof null === 'object', so
the null-check that guards against failed unions doesn't catch it).
Side-effect imports here force Meteor to bundle them.
A full recreate() over 7k+ subscriptions takes ~20 minutes even in a
worker thread — fine for not blocking the site, but means a new zone
takes ages to show up on the map. Since union only grows when adding a
circle, a new subscription can just be merged into the union already
stored (one turf.union call) instead of rebuilding the whole chain.
changed/removed still trigger a full recreate() (union can't be
"subtracted" from safely), and incrementalAdd falls back to a full
recreate whenever the stored count isn't exactly one behind what's
expected, rather than risk merging into stale state. Both paths now
share one serialized queue (FIFO for adds, coalesced for recomputes)
so they never race on the same stored union.
The previous fix (yielding to the event loop between turf.union calls)
was not enough: once the merged polygon gets complex with thousands of
subscriptions, a single turf.union call can itself take seconds, and
yielding between iterations doesn't help when one iteration alone
blocks that long. Confirmed in staging: the site was still fully
unresponsive (Cloudflare 524, curl hanging 2+ minutes, healthcheck
failing) while a recompute ran.
Validation/decoration (addNoisy/noNoisy, cheap) stays on the main
thread; the actual circle+union chain now runs in a worker_thread
(private/workers/unionWorker.js, plain CommonJS so meteor build copies
it verbatim instead of compiling it) so the main event loop serving
DDP/HTTP is never blocked by it, regardless of how slow any single
turf call gets.
Every subscription add/change/remove recomputed the full geo union
over all 7k+ subscriptions with a synchronous turf.union chain,
freezing the single Node event loop (and thus DDP/HTTP) for minutes.
The same recompute also runs at startup ("Subs union outdated"),
so every restart froze the site too.
calcUnionAsync yields to the event loop periodically during the
union chain, and subsUnion.js now fires recomputes without blocking
Meteor.startup or the observer callbacks, serializing overlapping
triggers instead of stacking them.
GlitchTip is behind Cloudflare, which answers the browser's cross-site ingest
POSTs with 503 (no CORS headers -> 'Failed to fetch'); server-side requests
pass. The browser SDK now posts envelopes same-origin to /sentry-tunnel, and
the server forwards them to GlitchTip:
- imports/startup/server/sentryTunnel.js: WebApp handler that relays the raw
envelope to <dsn>/api/<project>/envelope/?sentry_key=<key> over IPv4
(family:4 avoids Happy-Eyeballs picking Cloudflare's unroutable IPv6 on
IPv4-only hosts). GlitchTip authenticates by the sentry_key query, so the
key is derived from the DSN and passed explicitly.
- client ravenLogger: tunnel: '/sentry-tunnel'.
Verified: POST /sentry-tunnel -> 200 (envelope reaches GlitchTip).
flowkey:raven (Sentry legacy SDK) is dead on Meteor 3 and, with the old
sentry.comunes.org DSN unreachable, spammed the boot log with 502s on every
exception. Replaced by the modern SDKs (8.x) behind the same ravenLogger.log()
facade so the 6 call sites are unchanged. Empty DSN -> plain console logger
(same behavior as before).
End-to-end verified against a real backend: GlitchTip deployed on aaron
(Sentry-compatible, fits in ~1.5GB vs Sentry's 16GB), fronted at
sentry.comunes.org. A test exception sent from the dev server appeared in
GlitchTip within seconds. REST smoke byte-identical (12/12).
alanning:roles removed (client crashed the whole bundle; plain user.roles
checks instead), selaias:cookie-consent replaced by in-repo React banner,
publish-performant-counts vendored with countAsync, Meteor.autorun dropped
(App)/Tracker.autorun (FiresMap), and the map publications not covered by
the REST smoke (activefiresmyloc, activefiresunionmyloc, fireAlerts,
oauth.verifyConfiguration) converted to async APIs.
Verified in browser: / and /fires render with map + cookie banner, zero
uncaught exceptions. REST smoke byte-identical (12/12).
fixtures: drop @cleverbeagle/seeder (sync Mongo, unmaintained) for a small
idempotent async seeder (same dev accounts). sitemaps: drop dead
gadicohen:sitemaps for a WebApp.connectHandlers /sitemap.xml with the same
static page list (the per-fire section was already disabled). Verified on
dev boot: 6 users ensured, sitemap serving; REST smoke byte-identical.
Major milestone: the Meteor 3.1 server boots and runs against dockerized
MongoDB 7, and the REST framework responds (status/uptime OK).
- collection2 v4: import its eager entry 'meteor/aldeed:collection2/static.js'
from new eager files server|client/00-collection2-init.js so main.js runs and
patches attachSchema before collections load (v4 is fully lazy, no main module).
- fixtures.js disabled: @cleverbeagle/seeder uses sync Mongo, no Meteor 3 support
(dev fixtures, not needed for smoke). Debt.
- sitemaps.js disabled: gadicohen:sitemaps@0.0.17 pre-0.9 API, no 'sitemaps'
global on Meteor 3. Debt (SEO).
- Vendored restivus patched for async endpoints: route handler is async and
awaits _callEndpoint; _callEndpoint awaits the action + auth/role checks.
NEXT: Rest.js endpoints + helpers (countRealFires/firesUnion/whichAreFalsePositives/
fireFromHash/subscriptionsInsert/upsertFalsePositive) still use sync Mongo
(.count()/.findOne()/.fetch()/.insert()/.upsert()) -> convert to *Async, then
smoke test against Mongo 7.
Reaches Meteor 2.3 building + running against Mongo 3.2, REST smoke test
byte-identical to the 1.6.1.1 baseline (all 12 Flutter endpoints green).
Dead/abandoned Atmosphere packages removed or replaced (the upgrade blockers):
- arkham:comments-ui (no Meteor 2.x build; pinned accounts-password@1.x):
reimplemented the fire-page comments as a React feature:
imports/api/Comments/ (collection, server methods, publication, media
analyzers, new-fire-comment email) + imports/ui/components/Comments/CommentsBox.
Comment text now rendered as safe plain text + image/youtube embed (was
markdown-to-HTML). Wired into Fires.js; sitemaps.js + migration v11 updated
to the new collection. Old Blaze/startup comments files deleted.
- nimble:restivus (REST API; pinned accounts-password@1.3.3, CoffeeScript source
that crashes this build host): vendored as a local package
packages/nimble-restivus with the .coffee precompiled to plain JS and the
accounts-password constraint loosened to 2.x. REST behavior unchanged
(verified by smoke test). This also dropped the entire iron:router stack.
- maximum:server-transform (+ peerlibrary:*, meteorhacks:zones/inject-initial):
unused; removal broke Meteor.publishTransformed in FalsePositives publications
-> replaced with plain Meteor.publish (no transform was configured).
- less, markdown (no source files), and the dead test stack
(meteortesting:mocha, practicalmeteor:chai, xolvio:cleaner) which transitively
pulled coffeescript@1.0.17 — the build plugin that crashed meteor-tool
(node_contextify assertion) on this machine. Removing it unblocked the build.
- fourseven:scss 4.5.4 -> 4.14.1 (node-sass 4.5.3 doesn't build on node 12).
npm-mongo driver at 2.3 is 3.9.x — still compatible with the production Mongo
3.2 replica set.
The push/email notification processing now lives in the tcef-notifications
service (fases 1a-1c). Removed from the web:
- imports/startup/server/notificationsObserver.js (observe Notifications -> processNotif)
- imports/modules/server/notificationsProcess.js (node-gcm push + notif emails)
- the 'Process pending notif' SyncedCron job in cron.js
- node-gcm dependency (dead Google API since jun-2024)
subsUnion.js is KEPT: it feeds the subs-public-union SiteSettings that the REST
API (status/subs-public-union) serves to the Flutter app; it is not part of the
notifications pipeline.
DEPLOY ORDERING (see UPGRADE.md): production must not run this build until the
tcef-notifications service is emitting in prod — otherwise notifications stop.
REST smoke test green (byte-identical).
Adds a standalone Node smoke test (smoke/) that exercises every REST endpoint
consumed by the Flutter app against a seeded local dev server and compares
responses to committed snapshots. This is the safety net for the Meteor upgrade:
it must stay byte-identical after every escala.
- smoke/seed.js: deterministic Mongo seed (fixed ObjectIds, geo indexes)
- smoke/run.js: calls endpoints, normalizes volatile fields, diffs snapshots
- smoke/smoke.sh: seed + run wrapper
- smoke/snapshots/: baseline captured on Meteor 1.6.1.1
- IPGeocoder.js: degrade gracefully when MaxMind DB absent (dev boot)
- settings-development.json: dev-only internalApiToken to enable the REST API
- .meteorignore: exclude smoke/ from the Meteor server build
Retira import './segfaults' (workaround de crash obsoleto) y añade
leaflet-workaround.js (shims global window/document/navigator para
render de Leaflet en servidor). Ajustes menores en publications de
FalsePositives y subsUnion.
