ShapeTalk lets you search time-series data using natural language and sketches

Finding specific patterns in time-series data — think stock market trends, climate anomalies, or heart-rate irregularities — is notoriously hard when you can only describe them vaguely. Existing tools demand either an exact sketch or a rigid set of filters, which breaks down for fuzzy or composite patterns. ShapeTalk, introduced in a new arXiv paper by Sun, Chen, Guo, Yao, and Liu, tackles this head-on by treating natural language and sketching as complementary, not fused, inputs.

At its core is an LLM-based semantic parser that converts free-form queries like “a sharp drop followed by a slow recovery” into interpretable, editable shape-feature constraints. You can then refine the result by sketching over the actual data, and the system keeps both modalities in sync through a shared visual context and synchronized views. The team validated ShapeTalk with two usage scenarios and a formal user study, showing that language works well as an entry point while sketching fills in the gaps when text alone isn’t enough. For professionals who wrangle time-series data daily, this could finally make pattern search feel intuitive.