I have actually tried this and it works. Boot up from your USB stick, work in it, save the work back to the pen drive, exit out. Oh, also all the linux flavors now provide live cd/dvd. Here is a very helpful article on how to get ubuntu onto a pen drive:

http://www.pendrivelinux.com/usb-ubuntu-710-gutsy-gibbon-install/

Collecting data is only the first step towards wisdom, but sharing data is the first step toward community — Linux