These are not JIT compilers. They are AOT compilers. Code gets immediately compiled to machine code - either automatically or on direct request by the user.

But then what are the steps for make it real-able? I can see, like this project, the idea to compile everything and run, but the incremental aspect of a real is what I can't see how make it.

It's actually quite simple: all you need to do is to have your compiler be able to compile things smaller than the whole program. In Lisp, the unit of compilation is function (I think, lispm will correct me if I'm wrong), in Forth it's a word, in Prolog a rule and in CoffeeScript expression.

If you have that, the only thing that's left is being able to inject the compiled code into the running REPL process. This is more complicated nowadays because of restrictions placed on memory by the OS, ie. marking some segments of memory as executable and non-executable, but fundamentally compiled code is just a chunk of bytes, so "loading" here means allocating space for it and placing a pointer to the first byte anywhere reachable by the rest of the program, probably in some kind of a symbol table. The usual language rules take care of searching for the symbol's definition, then the implementation arranges the stack (or does anything else it wants with regards to arguments) and makes execution jump to the compiled code.

If you used GDB, you'd notice that it's possible to call natively compiled functions from its prompt, as well as to examine the symbols at the break-point (at least the ones not compiled away). GDB prompt is a REPL itself, although it only supports simple expressions IIRC, and the compilers of supported languages are written to work on the file level, not on the smaller chunks of code. But if you design a new language and implement a compiler for it, there's nothing stopping you from re-using compiler internals when writing a REPL.

The difference between JIT and AOT compilation is that with JIT you first execute the code, then compile it, while Ahead-of-Time (as the name implies) compiles code first and executes it later. What Lisp or Forth do is AOT compilation; for example, SBCL (IIRC) is a compiler-only implementation of Lisp, so it couldn't do JIT even if it wanted to, because there's no way for it to execute the code before it gets compiled. It - as mentioned - has a proper REPL and you'd be hard-pressed to find a difference in its look&feel compared to Python's.

BTW, writing an interpreter is very similar to writing a compiler: after lexing and parsing code into an AST you walk it, and then either perform required actions directly (that's an interpreter) or emit the code which will perform those actions when handed to a lower-level interpreter (like the CPU, or a VM). If you design your language from the beginning for it, you should be able to write both at the same time without much trouble. As an example, my favorite non-Racket Scheme implementation, Chicken Scheme, has both an interpreter and AOT compiler, with extensive support for dynamic behavior (like eval) in both compiled and interpreted code.

Anyway, this is my understanding, although I'm yet to write a compiler, so I'm not too sure about all the details, but the general outline should be more or less correct. I hope :)

EDIT: there was a book (which was not SICP) which taught writing interpreters and compilers and the relation between them, but I've been trying for half an hour now to recall what it was without success :( I should have the pdf somewhere, will try to dig it up later.

> If you have that, the only thing that's left is being able to inject the compiled code into the running REPL process.

This is alike JIT, where you MMAP and the mark that as executable?

Now thinking a little, could be easier using DLLs/dynamic linking, but how model the API is something I can't yet figure.

P.D: I think TinyCC is able to compile on memory, but I'm using rust for my project and probably the gymnastics of this are too much for now.