There is evidence - various psychology research projects that actually tested people with an without task-switching.
