Unfortunately I think that the mirrorless cameras may break your budget - best prices I have seen for the NEX 3 with a 'normal' lens is $599 - and you really will need at least a memory card and some sort of cover/bag for the thing.
The 'Big Three' all have a DSLR camera that would fit your budget (and leave room for buying some sort of bag , memory and a spare battery) and be much more capable than a Point and Shoot (quoting 'new' prices from Adorama here):
Canon XS - $495
Nikon D3000 - $495
Pentax K-x - $520
I bought an XS late last year mostly because I've always owned Canon and although I am VERY happy with it - after reading the reviews of the Pentax K-x I would have to say that it looks like quite a good camera, and I wish I would have given it a look. I'm sure that there are people who would say very nice things about the D3000.
I found that trying the XS and a D3000 in a local store really helped me decide between them - ie which one 'felt right' to me. I strongly recommend going to a store where you can actually hold these things and take a shot or two or three.
Even better - if you had any friends that have DSLRs, invite them to dinner and tell them that the 'charge' is that they need to bring the camera so you can give it a try around your house