Notes From Trying to Cache a Healthcare Chatbot
·23 min read A field log from a few months spent trying to put a semantic cache in front of a maternal-health chatbot. It never shipped to users — and the reasons it didn't are more useful than a success story would have been: in a clinical domain, 'similar' is not 'safe to reuse', and almost nothing we tried could tell the two apart at a useful hit rate.
AI SystemsSemantic CacheLLMsEmbeddingsHealthcareProduct