It's not just the taxes. Changes in regulatory restrictions on property use are takings because changing what a property owner can do with their property limits their opportunities to realize value from their property. There were a number of cases from the Supreme Court dealing with this idea. IANAL, but I believe that while annexation meets the due process requirements of the law, it still involves taking property rights from the annexed.