splitting and joining huge files

Today I wanted to sftp a 10Gb file over the wire. Knowing that the transfer would fail in the middle of it, I decided to split this file into small pieces beforehand. Below I think is the easiest way on osx.

// to split my huge file into 100mb files, having the prefix "bak_"
split -b 100m my_huge_file.ext bak_

// to re-construct my huge file
cat `ls bak_*` > my_huge_file2.ext

After running the split command, you will see some files named bak_aa, bak_ab, etc in your directory. These are the chucks of your original huge file. So just transfer these small bak_* files over the wire.

sftp doesn’t have mput… sigh…

Also the original file name is gone missing in the process… sigh… That’s unfortunate. What I decided to do is to write up a small readme file to store the above two commands, so later on I know what output filename to use when running the second command.