Illya Gerasymchuk profile photo
Illya Gerasymchuk
Financial & Software Engineer

Who would've thought that reordering 5 billion rows in a DuckDB table would need over 300GB in temporary storage on your hard drive

User
Illya Gerasymchuk

2026-03-05 19:19

Who would've thought that reordering 5 billion rows in a DuckDB table would need over 300GB in temporary storage on your hard drive

Luckily there is `SET temp_directory`, so I'm running ORDER BY again pointing it to a medium that should meet DuckDB's storage needs

I still believe my approach is feasible

Who would've thought that reordering 5 billion rows in a DuckDB table would need over 300GB in temporary storage on your hard drive
πŸ’¬