Hi,
Personally, I dislike FileVault, as it causes more issues than it prevents. It basically encrypts the drive, and it needs to unencrypt when you login and access files. Sadly, many folks have lost their data due to FileVault itself becoming corrupted, which is why I never recommend using it. Some businesses do require it, which can cause issues like you are experiencing.
Unfortunately, I don't know of any way to optimize it, as there are any controls for it. In addition, if you are running low on free space, it can cause slowness, as FileVault needs extra space to encrypt/decrypt files as they are accessed. If free space gets critically low, then you may not be able to login at all.
C