I enjoyed the heck out of this conversation between Hugo Browne-Anderson and Jason Liu on Hugo’s podcast, Vanishing Gradients. It took me a while to realize that Jason is the “Pydantic is all You Need” guy. I’ll comment on that discussion later.
What’s really interesting here is that Jason has applied background in using LLMs in anger to create production systems. There’s repeated back and forth about RAG vs. Long Context, vs. Fine Tuning. Which is a big taking point in the AI space right now. Around 12:05 Jason gets into the trade offs between latency and compute. This theme pops up over and over. In short, Jason feels RAG is here to stay, and he makes a compelling case for that.