node.js - Run NodeJS event loop / wait for child process to finish -

i first tried general description of problem, more detail why usual approaches don't work. if read these abstracted explanations go on. in end explain greater problem , specific application, if rather read that, jump "actual application".

i using node.js child-process computationally intensive work. parent process it's work @ point in execution reaches point must have information child process before continuing. therefore, looking way wait child-process finish.

my current setup looks this:

importantdatacalculator = fork("./runtime"); importantdatacalculator.on("message", function (msg) {     if (msg.type === "result") {         importantdata = msg.data;     } else if (msg.type === "error") {         importantdata = null;     } else {         throw new error("unknown message datagenerator!");     } }); 

and somewhere else

function getimportantdata() {     while (importantdata === undefined) {         // wait importantdatagenerator finish     }      if (importantdata === null) {         throw new error("data not generated.");     } else {         // should have proper data         return importantdata;     } } 

so when parent process starts, executes first bit of code, spawning child process calculate data , goes on doing it's own bit of work. when time comes needs result child process continue calls getimportantdata(). idea getimportantdata() blocks until data calculated.

however, way used doesn't work. think due me preventing event loop executing using while-loop. , since event-loop not execute no message child-process can received , condition of while-loop can not change, making infinite loop.

of course, don't want use kind of while-loop. rather tell node.js "execute 1 iteration of event loop, me". repeatedly, until data need received , continue execution left of returning getter.

i realize poses danger of reentering same function several times, module want use in nothing on event loop except waiting message child process , sending out other messages reporting it's progress, shouldn't problem.

is there way execute 1 iteration of event loop in node.js? or there way achieve similar? or there different approach achieve i'm trying here?

the solution think of far change calculation in such way introduce yet process. in scenario, there process calculating important data, process calculating bits of data important data not needed , parent process these two, waits data 2 child-processes , combines pieces when arrive. since not have computationally intensive work itself, can wait events event loop (=messages) , react them, forwarding combined data necessary , storing pieces of data cannot combined yet. introduces yet process , more inter-process communication, introduces more overhead, avoid.

edit

i see more detail needed.

the parent process (let's call process 1) process spawned process (process 0) computationally intensive work. actually, executes code on don't have control, cannot make work asynchronously. can (and have done) make code executed regularly call function report it's progress , provided partial results. progress report send original process via ipc.

but in rare cases partial results not correct, have modified. need data can calculate independently normal calculation. however, calculation take several seconds; thus, start process (process 2) calculation , provide result process 1, via ipc message. process 1 , 2 happily calculating there stuff, , corrective data calculated process 2 finished before process 1 needs it. 1 of results of process 1 needs corrected , in case have wait process 2 finish calculation. blocking event loop of process 1 theoretically not problem, since main process (process 0) not be affected it. problem is, preventing further execution of code in process 1 blocking event loop, prevents ever receiving result process 2.

so need somehow pause further execution of code in process 1 without blocking event loop. hoping there call process.runeventloopiteration executes iteration of event loop , returns.

i change code this:

function getimportantdata() {     while (importantdata === undefined) {         process.runeventloopiteration();     }      if (importantdata === null) {         throw new error("data not generated.");     } else {         // should have proper data         return importantdata;     } } 

thus executing event loop until have received necessary data not continuing execution of code called getimportantdata().

basically i'm doing in process 1 this:

function callback(partialdatamessage) {     if (partialdatamessage.needscorrection) {         getimportantdata();         // use data correct message         process.send(correctedmessage); // send corrected result main process     } else {         process.send(partialdatamessage); // send unmodified result main process     } }  function executecode(code) {     run(code, callback); // callback called time time when code produces new data     // call synchronous, run blocking until calculation finished     // if reach point done     // way pause execution of code not return callback  } 

actual application/implementation/problem

i need behaviour following application. if have better approach achieve feel free propose it.

i want execute arbitrary code , notified variables changes, functions called, exceptions occur etc. need location of these events in code able display gathered information in ui next original code.

to achieve this, instrument code , insert callbacks it. execute code, wrapping execution in try-catch block. whenever callback called data execution (e.g. variable change) send message main process telling change. way, user notified execution of code, while running. location information events generated these callbacks added callback call during instrumentation, not problem.

the problem appears, when exception occurs. want notify user exceptions in tested code. therefore, wrapped execution of code in try-catch , exceptions out of execution caught , send user interface. location of errors not correct. error object created node.js has complete call stack knows occurred. location if relative instrumented code, cannot use location information is, display error next original code. need transform location in instrumented code location in original code. so, after instrumenting code, calculate source map map locations in instrumented code locations in original code. however, calculation might take several seconds. so, figured, start child process calculate source map, while execution of instrumented code started. then, when exception occurs, check whether source map has been calculated, , if hasn't wait calculation finish able correct location.

since code executed , watched can arbitrary cannot trivially rewrite asynchronous. know calls provided callback, because instrumented code so. cannot store message , return continue execution of code, checking during next call whether source map has been finished, because continuing execution of code block event-loop, preventing calculated source map ever being received in execution process. or if received, after code execute has finished, quite late or never (if code execute contains infinite loop). before receive sourcemap cannot send further updates execution state. combined, means able send corrected progress messages after code execute has finished (which might never) defeats purpose of program (to enable programmer watch code does, while executes).

temporarily surrendering control event loop solve problem. however, not seem possible. other idea have introduce third process controls both execution process , sourcemapgeneration process. receives progress messages execution process , if of messages needs correction waits sourcemapgeneration process. since processes independent, controlling process can store received messages , wait sourcemapgeneration process while execution process continues executing, , receives source map, corrects messages , sends of them off.

however, not require yet process (overhead) means have transfer code once more between processes , since code can have thousands of line in can take time, move around little possible.

i hope explains, why cannot , didn't use usual "asynchronous callback" approach.

adding third ( :) ) solution problem after clarified behavior seek suggest using fibers.

fibers let co-routines in nodejs. coroutines functions allow multiple entry/exit points. means able yield control , resume please.

here sleep function official documentation that, sleep given amount of time , perform actions.

function sleep(ms) {     var fiber = fiber.current;     settimeout(function() {         fiber.run();     }, ms);     fiber.yield(); }  fiber(function() {     console.log('wait... ' + new date);     sleep(1000);     console.log('ok... ' + new date); }).run(); console.log('back in main'); 

you can place code waiting resource in function, causing yield , run again when task done.

for example, adapting example question:

var pausedexecution, importantdata; function getimportantdata() {     while (importantdata === undefined) {         pausedexecution = fiber.current;         fiber.yield();         pausedexecution = undefined;     }      if (importantdata === null) {         throw new error("data not generated.");     } else {         // should have proper data         return importantdata;     } }  function callback(partialdatamessage) {     if (partialdatamessage.needscorrection) {         var thedata = getimportantdata();         // use data correct message         process.send(correctedmessage); // send corrected result main process     } else {         process.send(partialdatamessage); // send unmodified result main process     } }  function executecode(code) {     // setup child process calculate data     importantdatacalculator = fork("./runtime");     importantdatacalculator.on("message", function (msg) {         if (msg.type === "result") {             importantdata = msg.data;         } else if (msg.type === "error") {             importantdata = null;         } else {             throw new error("unknown message datagenerator!");         }          if (pausedexecution) {             // execution waiting data             pausedexecution.run();         }     });       // wrap execution of code in fiber, can paused     fiber(function () {         runcodewithcallback(code, callback); // callback called time time when code produces new data         // callback synchronous , blocking,         // yield control event loop if has wait child-process finish     }).run(); } 

good luck! better solve 1 problem in 3 ways solving 3 problems same way. i'm glad able work out worked you. admittingly, pretty interesting question.