Speed really depends on the connection type(plan) you have :p
But in FileZilla, it assumes that you download and upload only small files like images, text files etc; so it sets the limit of download and upload to something like 5 or 10 simultaneous download/upload.
So the concept: You’ll get water if you Digg deeper at one place rather than to Digg small small pits at 10 different locations!
Set the download limit to 2.
Goto Edit -> Settings
Click on Transfer. Change the download limit to 2. Click OK
Now start the download again.
The transfer of files from server to your client machine will be much faster.