Never been a problem for me, but am not remotely brown and have global entry.
That has been my plan if challenged: I work in healthcare, and it could violate HIPAA and I could go to jail or lose my job. TalK to the company lawyer, get a warrant, or both.