Came here to recommend Amelia/Fernandina before I saw that OP was wanting to fly into ORL.
I love that area. There area bunch of good restaurants. 29 South has really good modern Southern. My boss is kind of a foodie and he takes weekend trips over from Destin a few times a year primarily to eat here - loves it. Espana, the tapas place, is also really good. They had a daily special shrimp and chorizo skewers in some lemon sauce that was outstanding, and they used to make it off-menu for me. Cafe Karibo is a good lunch spot. I was never that impressed with Brett's entrees, but they have a pretty nice bourbon/scotch selection and sipping a nice whisk(e)y in a rocking chair on the patio watching the sunset is tough to beat. Green Turtle Tavern is one of my favorite bars anywhere. I just love the vibe there. Grab a local brew and play darts on the patio listening to live, local music.
My wife really liked the walking ghost tour, but she's super into that stuff. There's enough "haunted" bars along the way that I was able to stay pretty entertained as well.
Plenty of beach access along Amelia, but it's closer to Kool's Ponte Vedra pic than SoWal, Destin, et al. There's a few river mouths in the area as well, so after a good rain to the north, the outgoing tide especially can get even dingier.
Fort Clinch is really cool if you're into that, and even the kids will dig fossil hunting. The Naval Sub Base to the north keeps the channel dredged for their vessels, so they are constantly sucking up really old sediment and spoiling it on the shore. I found a bunch of partial megalodon teeth, lots of small shark teeth, and several other cool fossils. Especially around sunrise, the wild horses can be seen roaming along the shore of Cumberland Island to the north.
Taking the ferry to Cumberland to check out the National Seashore and ruins of Andrew Carnegie's mansion is a pretty cool trip to make too. The kids might not get too into it, but they'd probably enjoy a boat ride either way.
Anyhow, maybe not for OP, but a really neat area to check out sometime and a pretty quick jaunt from JAX. Lots of good eats, decent beaches, good golf, history, kitschy boutiques, and just an all-around cool part of Florida to check out.
