I was in an around San Diego for two years. I go back every now and then.
Proximity to the zoo should not really be a big deal. I mean, if you can stay on the beach, why wouldn't you?
The three areas that would be good for tourists would include La Jolla, Del Mar, and maybe Torrey Pines. There are ample choices for housing in all of those areas. I am a huge fan of Torrey Pines beach.
Downtown San Diego is ok, but really not the best. The Gaslamp District is an armpit. Full of tourists and is more like Sixth Street than true San Diego.
When my kids were that age, the Zoo was a big hit. The Wild Animal Park is also worth the trip. It is indeed outside the city but it is a unique experience, especially for younger children (the safari train, the house of butterflies, the shows, etc). Legoland may also be of interest. My kids loved it, but it might be a bit of a stretch for 18month old.
Disneyland is not that far. And it is a great place for a young kid.
There is also a pretty cool aquarium in La Jolla.
San Diego is very diverse. You have mountains, dessert, beaches, etc. Not sure that Julian (the mountain town) is anything special, but if you want a change of pace it is there.