My understanding is that XHTML doesn't have any "non-semantic" elements. You can still use non-semantic HTML4 elements in XHTML Transitional mode, but they are not part of XHTML per se. The following list is off the top of my head, so I have probably missed some but the main ones to avoid are listed.

b i tt u strike big small sub sup center font

IMHO I can't imagine why you would use <br /> tags unless maybe you were generating content dynamically with PHP.
<hr /> is fine when used properly and not just as decoration.