TableRAG: Million-Token Table Reasoning with Language Models

Recent advancements in Large Language Models (LLMs) have expanded their capabilities in table reasoning. However, traditional methods that feed entire tables into LLMs face scalability issues due to the context length constraints of LLMs. These challenges are compounded by increased computational costs and potential degradation in reasoning capabilities known as the "Lost-in-the-Middle" phenomenon. To address these limitations, we introduce TableRAG, a retrieval-augmented generation (RAG) framework for LLM-based table reasoning. This framework combines schema retrieval, which identifies crucial columns and their data types, with cell retrieval, focusing on extracting essential keywords and information. This dual approach allows for efficient data encoding, significantly reducing token costs and avoiding information loss. We evaluated TableRAG's performance using new benchmarks derived from the Arcade and BIRD-SQL datasets and expanded synthetic data from TabFact for multi-scale evaluations. The results confirm that TableRAG efficiently and effectively solves queries across various table sizes, demonstrating its potential as a scalable solution for LLM-based table reasoning.