Super excited to share our work on LOTUS, a query engine for reasoning over large corpuses of data with LLMs! Joint work w/ the amazing @sid_jha1, @matei_zaharia & @guestrin Read the paper: arxiv.org/abs/2407.11418 Try out the code: github.com/stanford-futur… 🧵👇

Can interpretability help defend LLMs? We find we can reshape activations while preserving a model’s behavior. This lets us attack latent-space defenses, from SAEs and probes to Circuit Breakers. We can attack so precisely that we make a harmfulness probe output this QR code. 🧵

🧬GWAS is fundamental in drug discovery, linking disease to genetic variants. However, studying rare and uncommon diseases with GWAS is hard due to the huge sample sizes required. How can we use AI to help GWAS with small cohorts? In a multi-year collaboration @GSK…

🧵LLMs are great at synthesizing info, but unreliable at citing sources. Search engines are the opposite. What lies between them? Our new paper runs human evals on 7 systems across the✨extractive-abstractive spectrum✨for utility, citation quality, time-to-verify, & fluency!

One of this year’s Weaviate highlights already - with plenty more to come before the year is over. I love seeing customers’ reactions when they upgrade to v1.27 and switch on ACORN. (To be fair, I reacted the same way when I first tried it)

Awesome to see ACORN delivering huge speedups for filtered vector search at Weaviate! Try it out and check out our full paper here: dl.acm.org/doi/10.1145/36…

Prior work has used LLMs to simulate survey responses, yet their ability to match the distribution of views remains uncertain. Our new paper [arxiv.org/pdf/2411.05403] introduces a benchmark to evaluate how distributionally aligned LLMs are with human opinions. 🧵

This new episode from the @DisseminatePod breaks down our recent work, ACORN, which provides a sota index for searching over unstructured and structured data using vector search with predicates! Thanks for the fun conversation @jwaudberry

📢 I am recruiting PhD students for Fall 2025 @ Georgia Tech! If you are interested in topics related to DB + AI (multimodal & vector databases, data management support for ML, etc.), apply and mention my name in your application: gradapp.gatech.edu/apply/

Tables are a goldmine for accurate, fresh, domain data that LLMs should be grounded in for RAG, factver/QA, text2sql: retrieval is key! We introduce 🎯TARGET, a benchmark for evaluating table retrieval! Code,data,paper: target-benchmark.github.io Thinking bm25 ftw? Think twice..🧵

This must be tweeted every Halloween. I don’t make the rules

We now have a super cool demo of Table Augmented Generation (TAG) that lets you explore important datasets (e.g., US Election Donations). Try it now! (while we still have LLM inference credits) The TAG Project combines the language reasoning and world knowledge of LLMs with…

📢Annoucing EDLM, our brand-new Energy-based Language Model embedded with Diffusion framework! Key results: 1. We (for the first time?) almost match AR perplexity. 2. Significantly improved generation quality. 3. Considerable sampling speedup without quality drop. 🧵1/n

Many providers offer inference APIs for the same models: for example, there were over nine Llama-3 8B APIs in Summer 2024. Do all of these APIs serve the same completion distribution as the original model? In our new paper, ✨Model Equality Testing: Which Model is This API…

This is so cool! Check out this demo from @_asimbiswal for asking questions over large tables, like election contributions🗳️I have so many questions to ask👀 ✨And it's powered by Text2LOTUS! NL questions are converted to LOTUS programs for optimized LLM-powered data processing!

Awesome to see 10x faster hybrid search using ACORN in production! Really exciting release from @weaviate_io

DocETL is our agentic system for LLM-powered data processing pipelines. Time for this week’s technical deep dive on _gleaning_, our automated technique to improve accuracy by iteratively refining outputs 🧠🔍 (using LLM-as-judge!)

AI has the potential to transform real-world domains. But can AI actually improve outcomes in live interactions? We conducted the first large-scale intervention of a Human-AI Approach that has statistically significant positive learning gains w/ 900 tutors & 1,800 K12 students.

While operating system architectures have stood still for the past 30 years, hardware has been getting faster and faster. Accessing disks or the network used to take many milliseconds, now it takes microseconds. What would it take to build an OS that can handle these speeds--a…

Surprised not to see many say this: OpenAI just declared the *end* of a paradigm, more so than the beginning of a new one. Scaling inference is powerful and interesting; same as two years ago. What's new? Declaring that scaling up the model itself & doing standard RLHF is over.

Want to learn how the fastest database query engines really work? 🚀 I like this paper because it goes out of its way to do a fair experimental comparison of two state-of-the-art query processing techniques: vectorization and data-centric code generation. These are two different…