WP-Filebase Settings-->File Browser; select your sort order for Categories and Files.
I personally used the "file_custom_cf1:Custom1" field (there is a 2 as well) so that can manually order my sort. I use 3 digit numbers such as 001, 020, 300, 875, etc... because on my one site I do not have more than 1,000 files. However, on my other site I have over 50,000 files and am struggling since this plugin broke with WordPress 4.2.2.