AI Safety

Ranking inkhaven bloggers: Update

Anthropic's Claude Code automates ranking of 20+ Inkhaven bloggers, with scores reaching +2.99σ for top posts.

Deep Dive

Independent researcher Sean Herrington has deployed Anthropic's Claude Code to create an automated ranking system for the Inkhaven writing residency, showcasing practical AI application in content analysis. The system scrapes blog posts from multiple platforms (each with different technical requirements) and assigns statistical scores based on performance metrics, with top posts like Natalie Cargill's "How to invent a disease" reaching +2.99σ and viv's pregnancy-related content scoring +2.75σ as the top author across 13 posts.

Herrington's implementation includes error margins (±0.18σ for top posts) and handles technical challenges like inconsistent scraping across different blogging platforms. The AI-powered system has been integrated into a searchable website interface and displayed on physical whiteboards, increasing resident engagement through transparent performance tracking. Analysis shows normal distribution of scores but no significant improvement in writing quality during the residency period, suggesting the scoreboard itself may drive future improvements.

The project demonstrates how Claude Code can automate complex data aggregation and analysis tasks that would typically require manual coding. By processing content from diverse sources and generating statistically meaningful rankings, this application shows AI's potential for creating real-time feedback systems in creative communities. The inclusion of error margins and distribution analysis adds scientific rigor to what might otherwise be subjective evaluation of writing quality.

Key Points
  • Claude Code automated scraping and ranking of posts from multiple blogging platforms with different technical requirements
  • Top post scored +2.99σ (Natalie Cargill) while top author viv achieved +2.75σ across 13 posts
  • System now includes error margins and searchable interface, increasing resident engagement through transparent scoring

Why It Matters

Shows how AI can automate complex content analysis and create real-time feedback systems for creative communities.