Yes, there's lots of folks doing this and currently it looks like combining results from an LLM based index and a standard text retrieval indexes (e.g. using BM25) may beat either alone. Note that you can add and search LLM derived vectors from within Elasticsearch: look up 'dense vector search' in the docs. Check out the Haystack conference next week for lots of discussion of current practices: https://haystackconf.com/2023/

Thanks!

I've looked at haystack a little bit and was wondering how involved it would be to set up for a research spike.

The LLM embedding approach has given me much better search results than Elasticsearch.

Cost is definitely an issue.