I think writing an effect library yourself is a tough ask, but some of them have gotten really, really good. And they get you things that are simply not possible with promise. Check out Effection if you want a more vanilla javascript syntax, or Effect if you're really into expressing things functionally.
Is it safe to just "stop calling next() on a generator?" like the post suggest?
To me that sounds like dropping the task on the floor. Specifically, this will not invoke any finally {} blocks:
More correctly, you should invoke `return()` on the generator. Otherwise, you won't provide execution guarantees. This is how Effection does it. There is no equivalent in async functions, so it sounds like the same problem would apply to the GC technique.
JavaScript generators are unmined gold! We've done a lot of usage of JavaScript generators as delimited continuations; using them to implement the classic shift/reset operations in JavaScript. https://github.com/thefrontside/continuation
I think the simplest expression of the idea of structured concurrency that I've come across is this: The scope of any concurrent process is the same as the lexical scope of the function that created it. This has profound implications.
Let's say, for example that I call `doTasks()` with 10 separate tasks. Before any of them can complete, one of them fails. What happens to the other 9? In your example, they are "leaked" because they will continue running, even though the scope that was awaiting their results in gone. Using a "worker pool" as in the example, it might go something like this:
async function doTasks(tasks = []) {
let workerpool = new WorkerPool();
try {
for (let task of tasks) {
workerpool.exec("my task", task);
}
let results = await workerpool.all();
// do stuff with results
} finally {
// no matter the outcome, nothing outlives this scope.
await workerpool.destroy();
}
}
It should be noted that cancellation, i.e. the ability to "destroy" a concurrent task is a necessary primitive for structured concurrency. You can think of it as the equivalent of automatically releasing memory at the end of a lexical scope.
This helps complete the picture, thanks. In my current use-case I am batching up precomputing some expensive calculations in a genetic algorithm which all needs to be finished before I can run the competition. If one of the phenotypes throws an error Promises.all will catch it and my program will continue and I can handle the error, but as for my workers, yup they will leak - and I can't stop that using promises / async / await without leaking some internals of my workers with some shared semaphore or something, which is so 1990's. So yours this was a perfect example that helped bring it all together.
Funnily enough after this thread I did some digging around and came across effection as well, I'd never come across the term "structured concurrency" before but it feels like a natural next step on what I'm trying to accomplish, so I'll definitely be keen to see more development in this area and maybe some first class language constructs at some point.
Thanks for taking the time to give an example and explain the concept in relatable terms.
> I'm not sure I buy the premise for the library's existence, at least with how the author describes it.
Just do a google search for "structured concurrency". These ideas did not come out of a vacuum, but instead from years of research in language communities ranging from C/C++ to Java, Swift, and Python. Effection is just these same ideas projected into the world of JavaScript.
