I was recently looking into an equivalent for V8 isolates. I'd like something like this for python, but it looks like micro VMs is my best bet here.
For anyone working in this field or having hands-on experience: is weave's ignite a choice if I'd want to execute long-running python scripts of users? I remember that there was a lot of overhead for I/O. Or should I just go with raw Firecracker like Stan does in this article?
The scripts would be long-running, but don't require much computational power. Just a few arithmetic operations on an array every second. The array is being fed in via WebSocket.
gVisor gives you all the container tooling, which may or may not be useful.
And, because I'm a shill, we actually shipped an API specifically for this kind of use case. So if you'd rather not build it all yourself, we can help: https://fly.io/blog/fly-machines/
I'm a fan of Firecracker, but your use case might be a better fit for plain old containers because the tooling is currently more mature. If it's the isolation that attracts you to Firecracker, gVisor is an option: https://gvisor.dev/.
For the open-source windmill project, we need to support sandboxing of typescript (deno) and python. For deno we could have just relied on v8 isolate and deno layer of isolation. But for Python we could not anyway so we had to come up with a common solution. We chose nsjail in the end and it works really well. All the config files are here: https://github.com/windmill-labs/windmill/tree/main/nsjail and this is how it is spawned from within the Rust worker: https://github.com/windmill-labs/windmill/blob/main/backend/...
Happy to expand more of my experience of making this work at scale.
If you take a look at the Skybison Python runtime, I would be happy to chat and help you poke around integrating it: https://github.com/tekknolagi/skybison
You want to run short lived Python in a trusted environment? Can you go into more detail about your specific use case? How much CPU, memory and IO do you need to do? Is it chatty over the life of the execution or does have all of its data up front?
The scripts would be long-running, but don't require much computational power. Just a few arithmetic operations on an array every second. The array is being fed in via WebSocket.