BIRD (BIg Bench for LaRge-scale Database Grounded Text-to-SQL Evaluation) is a cross-domain dataset that examines the impact of extensive database contents on text-to-SQL parsing. It contains over 12,751 unique question-SQL pairs, 95 big databases with a total size of 33.4 GB, and covers more than 37 professional domains including blockchain, hockey, healthcare, and education.

BIRD-SQL

Google's best model yet for coding performance and complex prompts, with better understanding and reasoning of world knowledge than any previous release. Features a massive 2 million token context window and the ability to call tools like Google Search and code execution.

Gemini 2.0 Pro

Next iteration of Gemini, released in three versions (Flash, Flash-Lite, Pro). Represents Google's state-of-the-art multimodal model, likely incorporating Mixture-of-Experts at unprecedented scale and targeting dominance in both text and image understanding.

Gemini 2.0 Flash

Best for cost-efficient performance. Better quality than 1.5 Flash, at the same speed and cost. Features a 1 million token context window and multimodal input.

Rank	Model	Provider	Score	Released	Type
1	Gemini 2.0 Pro	Google	59.3	2025-02-05	Multimodal
2	Gemini 2.0 Flash	Google	58.7	2025-02-25	Multimodal
3	Gemini 2.0 Flash-Lite	Google	57.4	2025-02-25	Multimodal

BIRD-SQL

BIRD-SQL Leaderboard

About BIRD-SQL

Methodology

Publication