I would have preferred to have 1 1/2 to two full days, with one overnight stay, for our visit. There are a lot of guided hikes, informational sessions, and even an IMAX movie to enjoy. I would have loved to walk more along the rim trail, too (we were only able to take the bus and stop at a couple of the overlooks). Unfortunately, because of our schedule and the route we took to get there and get home, we didn't arrive until late afternoon and had to depart about lunchtime the next day. The GC is kind of remote and has limited accessibility. If you are not from the area, I wouldn't recommend driving at night. But, if you also plan to spend time in Flagstaff or Sedona, it would be much faster to get to/from the GC than using PHX as a base.
As I mentioned above, I recommend the trail at the Hermit's Nest, simply because so few other visitors take the time to do it. Obviously we didn't hike all the way to the bottom of the canyon, but if you have three days and the weather is good, that is definitely something I would try to plan for. The Bright Angel Trail and the South Kaibab Trail go all the way to the bottom.
Also be sure to go out to one of the overlooks to watch the sunset. The GC puts out a newspaper that gives all the times/locations for daily activities, so you will know the time/place for things like guided tours, info sessions, and sunrise/sunset.
If you plan on camping overnight on the trail or at the bottom of the canyon or want to stay at Phantom Ranch, you have to make reservations well in advance (sometimes a year or more) and there is often a long waiting list. The National Parks Service tries to keep a log of people who go down in the canyon so they know when someone is missing and can better respond to emergencies. IIRC there are signs strongly discouraging people from trying to hike down and back up in the same day, but I don't believe it is expressly prohibited. You can also do mule rides into the canyon and back up.
Note: all of my comments are related to the South Rim, which is where is visitors center and most of the accomodations are. The North Rim is even more remote/difficult to access (7 hours from PHX) and is closed from Oct to May. Other than a day visit or camping, there is only one lodge inside the North Rim park.