The Founders were anti-gay?

Per TPM, Rep. Lamar Smith (R-TX), chairman of the House Judiciary Committee, bashed the Justice Department's decision not to defend the Defense of Marriage Act in court because they found a provision of the law unconstitutional. His rationale: "No one can seriously believe that the constitution's authors intended to create a right to same-sex marriage." Al-righty then. But by Smith's logic, we should also question the civil rights of woman and minorities, too. After all, the "authors" were not exactly keen on either group at the time, right congressman? I'll leave it to the 70s band Stealer's Wheel to put Smith into perspective: "It's so hard to keep this smile from my face / Losing control yeah, I'm all over the place / Clowns to the left of me, jokers to the right, here I am / Stuck in the middle with you."