I got it working with some help from a nice guy on the WordPress channel on IRC.
The shortcode we came up with does not allow any sorting, but it works:
<?php $value = get_field( "memos" ); echo do_shortcode('[wpfilebase tag=browser id="'. $value .'"]');?>
"memos" is the name of my ACF field.