Well if you try to remove a directory with multiple files rm will take a long time also some load on that server .
One solution is to rsync a empty directory .
mkdir empty_dir
rsync -a –delete empty_dir/ yourdirectory/
Also a perl can be made but the rsync is easy for me to remeber .
Some more information about this you can find here
https://serverfault.com/questions/183821/rm-on-a-directory-with-millions-of-files/328305#328305