Yes, but it wouldn't have looked good for him. It would've been a tacit admission of guilt on his part, and (to my mind about the man) that was unacceptable to him. As most politicians are masters of rationalization, he figured better to bullshit the way out of it than to refuse to answer. Again, that's my take on it.

Also, it could've led to a Contempt of Congress charge being leveled at him on top of everything else being thrown at him by the opposition. And the Contempt charge would've had real legs, quite unlike much of the innuendo and suspicion that was the rest of his opponents' arsenal.