Proper ground configuration is essential for a good data acquisition system. Most measurement systems such as data acquisition devices allow for many different types of ground configurations depending on the type of signal being acquired or measured. This flexibility is the source of confusion when deciding which configuration to use in each specific situation.
National Instruments tutorial Ground Loops and Returns teaches you to select the right configuration to use. Here is the material from the document related to ground loop problems in condensed format:
A grounded signal source is one in which the voltage signals are referenced to a system ground, such as earth or building ground. The most common examples of grounded signal sources are devices, such as power supplies, oscilloscopes, and signal generators that plug into the building ground through a wall outlet. The difference in ground potential between two instruments connected to the same building ground system is typically 10mV to 200mV, or even more (up to several volts at normal use and tens of volts during short circuit surges).
Single ended is the “default” configuration for most data acquisition devices, modular instruments, and stand-alone devices. Single-ended systems are very susceptible to ground loops. There are essentially two main types of Single-Ended measurement systems: Ground Referenced Single Ended (GRSE) and Non-Referenced Single-Ended (NRSE).
An ideal differential measurement system reads only the potential difference between the positive and negative terminals of the amplifier and thus it completely rejects common-mode voltages. However, practical devices are limited in their ability to reject common-mode voltage.
A grounded signal source is best measured with a differential or non-referenced measurement system.
The pitfall of using a ground-referenced measurement system to measure a grounded signal source is that grounding potential difference between signal source and measuring system causes a current called ground loop current to flow in the interconnection which can greatly affect measurements causing offset errors, especially when measuring low level signals from sensors.
Image source: http://zone.ni.com/devzone/cda/tut/p/id/3394