Argument list too long

While moving Seenly to a new server with more capacity, I had to upload a huge amount of tiny files. Trying to do ncftpput * in a directory that contains 27000 files each with a filename of 35 characters makes Bash cry, and spit out “Argument list too long”.

Xargs to the rescue:

echo * | xargs ncftpput

Amazingly, this works: Instead of trying to ncftpput everything at once, xargs executes ncftpput repeatedly with a group of files, until all files have been sent. Echo doesn’t throw an Argument list too long error because it’s a shell built-in. This basically works for every command you need to execute on a large number of files.

Watch out though, because the command isn’t interactive anymore. So if you get this error:

Error: gl_getline(): not interactive, use stdio.

It means it’s trying to get interactive, e.g. asking you for a password, while it’s not allowed. So make sure you set all required options using switches:

ncftpput -u user -p password hostname remote/path