Yeah, it changes a lot, at least for me. I mean, if you wrote a, say, torture scene about the DE's; the poor guy being cruci'd would be scared to death, and would see in greater detail due to panic, and the DE in question would probably be thinking what a jolly time he's having and whether he should have spaghetti or raviolis for dinner.

Gender do matters, but not as much as age, or the knowledge of that person. For example, I wrote a gen fic a while ago from Ron's POV, and while it was fun to describe a TV and a cable ended up being a 'noodle', it was also frustrating; and that happens with Harry's POV, we only get stuck with what he believes to be true, so a chapter like 'Spinner's End' could have never been watched from an objective (or possible, actually) point from his eyes. (and if you think about it, in that chapter we don't get an insight to someone's mind at all)