Officially you are supposed to tell Windows to Stop the USB key before pulling it out, so that Windows can flush its disk cache to USB. Unofficially you can set USB drive into a mode where disk cache will not be used, so you can pull out any time without stopping:
* Click "Safely Remove Hardware" USB icon in Taskbar.
* Check "Display Device Components" in dialog that comes up.
* Click the "USB Flash Disk Device", the middle one in the 3-level tree.
* Click Properties.
* Select Policies tab.
* Select "Optimize for Quick Removal" and click OK.