I have lived in Pagosa for over 6 years now. There is lots to do here year 'round.
Fishing- are you wanting to just drift a fly in the San Juan down town, or are you thinking of backcountry streams?
November is not going to be your best month, for sure. But it really depends on if you are talking about the beginning of the month or end. We typically have our first big snow around Halloween. Sometimes it sticks, sometimes it melts. Temps will be plummeting in Nov, but things won't be frozen over. Ice fishing won't start until late Dec or Jan. The San Juan downtown by the hot springs will still have open water and fish, but has also seen a whole season of fishermen. Anywhere else is likely going to be too snowy, too muddy, too slow, or too frozen. However- that is a great time of year (depending on water flows) to head down to the Quality Waters below Navajo and tangle with some big trout. There are some guided private waters worth checking out too (Ski & Bow Rack).
Hiking- again, it really depends on what you mean. Casual day hikes? Sure, you can find some accessible spots. Backcountry, CDT, or overnighters are most likely not going to be pleasant experiences in Nov. However, people aren't coming to Pagosa in Nov to hike.
What is there to do? The Hot Springs are fabulous that time of year. Sleigh rides with Buckaroos will be kicking off (skies or wheels). Of course, skiing at Wolf Creek will be loads of fun. There are some beautiful drives and shorter trials you could keep busy with. If we have snow already, renting snow shoes is always fun- or sledding tubes.
November can be full on winter, or still a bit of a tweener season here.