we had an on-prem splunk implementation and it was SOO SLOW.. it was built/managed by splunk and its consultants.
We finally got rid of it a few years later, but for the entire time we had it, it was a constant "round hole square peg" problems. Each time the consultants assured us Splunk could do what we needed, each time it could not.
Just coming back around to this, we also used Splunk consultants for their SIEM solution and the first one we got wasn't very good, but the second was amazing (I wish we could have hired her directly).
The guy we had help us tune our clusters after I rebuilt them all was also very good. Fortunately I'd done most everything by the books and we overkilled the nodes with hardware (we had some older hypervisor nodes lying around I stole for Splunk).
We finally got rid of it a few years later, but for the entire time we had it, it was a constant "round hole square peg" problems. Each time the consultants assured us Splunk could do what we needed, each time it could not.