I'm using GPT and its functions, which are basically JSON schemas for its output. It makes the formatting a lot more stable, but even then it's still doing tokenized completion and the function definition is just another aspect of the prompt.
I've done some productive collaborating with someone who only works at the prompt level, but you can't really hand off in my experience. You can do some things with prompts, but pretty soon you are going to want to change the pipeline or rearrange how data is output. Sometimes you just won't get a good response that is formatted the way you want, and have to accept a different function output and write a bit of code to turn it into the representation you want.
Also the function definition (i.e., output schema) looks separate from the prompt, but you absolutely shouldn't treat it like that; every description in that schema matters, as do parameter names and order. You can't do prompt engineering without being able to change those things, but now you will find yourself mucking in the code again. (Though I make the descriptions overridable without changing code.)
Anyway, all that just to say that I agree that code and prompt can't be well separated, nor can pipeline and prompt.
You can use the function definitions and few shot prompting (examples) to great effect together.
E.g. I was trying to build a classifier for game genres from descriptions. I could use the function definitions to easily define all the main genres and add subgenres as enums. That ensured that the taxonomy was pretty much always followed. But then I used examples to coerce the model into outputting in {'strategy':'turn-based'} format rather than {'genre':'strategy', 'subgenre':'turn-based'}. The tokens saved on that could then be used to do more classifications per gpt call, making the whole thing cheaper.
I've done some productive collaborating with someone who only works at the prompt level, but you can't really hand off in my experience. You can do some things with prompts, but pretty soon you are going to want to change the pipeline or rearrange how data is output. Sometimes you just won't get a good response that is formatted the way you want, and have to accept a different function output and write a bit of code to turn it into the representation you want.
Also the function definition (i.e., output schema) looks separate from the prompt, but you absolutely shouldn't treat it like that; every description in that schema matters, as do parameter names and order. You can't do prompt engineering without being able to change those things, but now you will find yourself mucking in the code again. (Though I make the descriptions overridable without changing code.)
Anyway, all that just to say that I agree that code and prompt can't be well separated, nor can pipeline and prompt.