We just returned from Madrid and it was incredibly amazing. Already ready to go back!
We managed to visit the Royal Palace, the Prado, the village of San Martin and two wineries there, an olive grove where the owners also produce their own olive oil, the town of Toledo, two walking tours, a flamenco show and lots of yummy food.
SkiMo, take the Huertas neighborhood food tour! It was three hours of non-stop eating and it hit spots that we hadn't managed to visit yet in our first 3 days. We also found a history walking tour through AirBnB experiences that was so-so. We had a lady from Boston who kept challenging our tour guide, so it made it less enjoyable. (For instance, we passed the oldest restaurant in the world, and she was trying to argue there was a place in Boston that was older. He told her it was certified by the Guinness Book of World Records, and her daughter looked up the date of the Boston location, and she still tried to qualify it.) I think it would be more enjoyable with a different group of people.
TexAggie79, you were spot on with your description of driving. We managed fine but we found it to be easily navigable once you get used to people cutting across lanes in the roundabouts.
TriAggie, you can definitely manage without renting a car. Parking is about 32 euros a day, so find a train or take a cab/uber in. I would skip the Prado unless you are REALLY into art. The Royal Palace is quite gorgeous. Buy your tickets online if you want to skip the long line. There are guided tours but the audio tour suited me quite well. There's a street behind the Plaza Mayor that is "rustic" but each spot specializes in one specific dish. We stopped in the mushroom one and the mushrooms were stuffed with chorizo and smelled heavenly. If I had more time, I would go tapas hopping back there.
For me personally, I didn't find Mercado San Miguel that exciting. It was crazy crowded, even on a weekday, and looked like tapas we could get anywhere else. I'm glad we went in, just to say we saw it, but it didn't really do anything for me. Although we had planned out where we wanted to eat, we never seemed to be in the right spot when we were hungry, so we just opted to eat wherever was close by and looked good. It worked out every time, we never had a bad meal in Madrid. I would recommend finding somewhere with dining outside in a plaza. You'll pay more than if you were to stand at a bar, but being out in the plaza with everyone buzzing about is fun.
The one thing I would highly recommend is finishing your night with churros con chocolate at Los Artesanos 1902 Chocolate. The churros are always fresh and the hot chocolate is amazing. We found it much better than San Gines. We also stayed in the Hotel Fransisco right down the street and enjoyed it.