usb1: Only clear pollfd notifiers if setPollFDNotifiers has been called.

On windows, libusb1 does not support poll fd notifiers, and emits a warning
if this is called. So only call it if the application decided to.