New AI Can Tell If a Sidewalk Is Social, Not Just Crowded

Two researchers, Liu Liu and Andres Sevtsuk, built an AI system that looks at street-level photos and judges how social a sidewalk really is. It is a "vision-language model" — AI that can look at a picture and describe what it sees in words. They fed it 102,514 panoramic street images of New York City, flattened into views facing the sidewalk, and asked it to note ten visible things about each person: are they standing or sitting, alone or in a group, walking or lingering.

Why bother? Because counting people tells you almost nothing useful. A sidewalk packed at 8 a.m. with commuters rushing to the subway is completely different from a sidewalk where people sit outside a cafe all afternoon. The old tools saw both as "busy." The new system produces a single number per street, a "Social Dwelling Index," that combines how people are grouped and whether they are staying put. The result was striking: foot traffic and social activity barely lined up at all. The busiest streets were often not the friendliest ones.

For you, this is mostly about the street outside your door. Cities spend real money deciding where to put benches, plant trees, widen sidewalks, or allow outdoor seating — and today those calls rely on expensive manual surveys or rough population counts. If a cheap AI can map where people genuinely gather, that money is more likely to land where it actually improves daily life. It could also quietly shape home values and small business traffic, since lively blocks attract shops and customers.

The catch is privacy. This system analyzes ordinary people in public without their knowledge, sorting them by posture and activity — and it flags "accessibility-sensitive populations," meaning people who may be elderly, disabled, or unhoused. It cannot know why someone is sitting on a curb, and it can misread what it sees. It is also a research paper, not a product, so the rules governing how cities might use it are still unwritten.