Any distracted driving may be harmful to anyone on the road. While it is illegal to drink and drive and to text and drive in most states, nowhere is it illegal to use a hands-free system of cell phone communication. Once seen as a safer alternative, according to research from the University of Utah, it can be classified under dangers on the road.
Many argue that a hands-free system is comparable to listening to the radio or an audiobook, but the University of Utah found that “listening to verbal material, by itself, is not sufficient to produce the dual-task interference” and “when a driver becomes involved in a cell-phone conversation, attention is withdrawn from the processing of the information in the driving environment necessary for safe operation of the motor vehicle.” Basically, the act of being in a conversation is putting the driver at risk.
“But drivers have conversations with passengers and are okay!” one may argue. However, passenger conversations and phone conversations differ in knowledge of traffic conditions. Whereas passengers can see the traffic and may adjust the conversation so that the driver can focus, someone who is not present in the car would not know the conditions and wouldn’t know when to adjust. Also, the passenger can be used as another set of eyes, whereas someone speaking over the phone cannot see an oncoming vehicle or other danger.
Although drivers are keeping their eyes on the road, the problem with hands-free technology is keeping their minds on the road. Double tasking is not safe for driving, even when the driver doesn’t need to avert his eyes from the road. Hands-free is not risk-free.

Source: http://www.psych.utah.edu/AppliedCognitionLab/cdir489.pdf