comunes/todos-contra-el-fuego-web
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(subsUnion): move geo-union math to a worker thread, not just yields
All checks were successful
build-image / build (push) Successful in 13m13s
Details

Browse source 
The previous fix (yielding to the event loop between turf.union calls)
was not enough: once the merged polygon gets complex with thousands of
subscriptions, a single turf.union call can itself take seconds, and
yielding between iterations doesn't help when one iteration alone
blocks that long. Confirmed in staging: the site was still fully
unresponsive (Cloudflare 524, curl hanging 2+ minutes, healthcheck
failing) while a recompute ran.

Validation/decoration (addNoisy/noNoisy, cheap) stays on the main
thread; the actual circle+union chain now runs in a worker_thread
(private/workers/unionWorker.js, plain CommonJS so meteor build copies
it verbatim instead of compiling it) so the main event loop serving
DDP/HTTP is never blocked by it, regardless of how slow any single
turf call gets.
This commit is contained in:
vjrj 2026-07-30 07:48:17 +02:00
parent 3d52e2f709
commit fbb746fba2
3 changed files with 75 additions and 48 deletions
Download patch file Download diff file Expand all files Collapse all files

68
imports/startup/server/calcUnionAsync.js
View file

@ -1,54 +1,28 @@
import tcircle from '@turf/circle';
import tunion from '@turf/union';
import ttrunc from '@turf/truncate';
import path from 'path';
import { Worker } from 'worker_threads';
const truncOptions = { precision: 6, coordinates: 2 };
// How many polygons to process before yielding to the event loop once.
const YIELD_EVERY = 100;
const yieldToEventLoop = () => new Promise((resolve) => { setImmediate(resolve); });
const WORKER_PATH = path.resolve(process.cwd(), 'assets/app/workers/unionWorker.js');
/**
* Circle-union of subscriptions, equivalent to map-common-utils' calcUnion(typeCircle=true),
* but split into a `for` loop that yields to the event loop every YIELD_EVERY polygons.
* The vendored calcUnion runs the same turf.union chain fully synchronously, which for
* thousands of subscriptions blocks Meteor's single-threaded DDP/HTTP handling for minutes
* (see plan-modernizacion notes on the subsUnion hang). This keeps the exact same geometry
* output while letting other requests interleave between chunks.
* Circle-union of already-validated, already-decorated subscriptions, run in a
* worker thread so the main event loop (DDP/HTTP) stays free regardless of
* how long any single turf.union call takes. subs must be an array of plain
* { location: { lat, lon }, distance } objects.
*/
const calcUnionAsync = async (subs, decorated) => {
const unionGroup = [];
for (let i = 0; i < subs.length; i += 1) {
const osub = subs[i];
try {
if (osub.location && osub.location.lat && osub.location.lon && osub.distance) {
const dsub = decorated(osub);
unionGroup.push(tcircle(
[dsub.location.lon, dsub.location.lat],
dsub.distance,
{ units: 'kilometers', steps: 144 }
));
} else {
// Expected for old telegram subscriptions predating the distance field
// (e.g. distance: null) — logged, not an error, safe to skip.
console.info(`Wrong element to do union ${JSON.stringify(osub)}`);
}
} catch (e) {
console.error(e, `Wrong element trying to make union ${JSON.stringify(osub)}`);
const calcUnionAsync = subs => new Promise((resolve, reject) => {
const worker = new Worker(WORKER_PATH, { workerData: { subs } });
worker.once('message', (msg) => {
worker.terminate();
if (msg.error) {
reject(new Error(msg.error));
} else {
resolve(msg.union);
}
// Deliberate: yield to the event loop periodically instead of blocking it.
// eslint-disable-next-line no-await-in-loop
if (i % YIELD_EVERY === 0) await yieldToEventLoop();
}
let unionTemp = unionGroup.length > 0 ? ttrunc(unionGroup[0], truncOptions) : null;
for (let i = 1; i < unionGroup.length; i += 1) {
unionTemp = ttrunc(tunion(unionTemp, unionGroup[i]), truncOptions);
// Deliberate: yield to the event loop periodically instead of blocking it.
// eslint-disable-next-line no-await-in-loop
if (i % YIELD_EVERY === 0) await yieldToEventLoop();
}
return unionTemp;
};
});
worker.once('error', (err) => {
worker.terminate();
reject(err);
});
});
export default calcUnionAsync;