I've only done one valve adjustment since putting my TT bars on, and the answer is yes. You may not have to completely remove them, as you only need a little bit more clearance to get the valve covers past them, but you will have to do most of the work of removing them. This probably adds 10-15 minutes to the job.
I am considering trying to space my bars out a little farther so they don't have to be removed, but that may be a winter project.