Thousands of records, containing personal financial information, were accidentally left at a pub when an unencrypted USB device was forgotten.