I have thought about this a lot as it is a dream of mine as well.
I owned a 40 acre tract in east Texas. I have since sold the property, since I just didn't get a chance to hunt out there that often once my job took me out of state. I agree with those saying to buy a place within a 3 hour drive unless you have a job that allows you to spend weeks at a time at the place.
I think land is a good investment long term, but it obviously isn't a very liquid investment. It could easily take a year or more to sell. You definitely don't want to get over extended investing in land. It also can be hard to obtain financing if you aren't willing to put at least 20% down.
I kind of go back in forth on if I will buy a decent tract of land later on in life. I would love to own a place large enough to practice solid managment, and do some small scale farming or ranching. Plus I want a place with enough space and game to not get bored. To me that would mean at least 500 acres and probably closer to 1000. I would want a fishing lake/live water, a place to duck hunt, big whitetail and some turkeys for the spring. To get a decent hunting place like that in Texas you are probably looking at well over $1 mil and probably closer to $3-5 mil range. That is a tremendous amount of cash for the average family. Especially since you can spend under $10k a year and be on a supreme property with trophy deer, year round access and you don't have to worry about the taxes, liability, fences, etc.
If i was a multi-millionaire I would buy a place in a heart beat, but for the average person it is just a huge investment and somewhat unrealistic.
I am leaning more towards maybe just having a place on a lake, the coast, or maybe a smaller tract of land in a prime location while still having the hunting lease to stratch my larger land itch. Not only is it much more realistical financially, but if you get tired of a place, or you hit some drought years, it is much easier to move when you are leasing.
If I did buy a large tract of land, I think it would be a dual agriculture lease so I could at least make a few bucks from leasing or working the land to offset the taxes and inherent costs. It's very hard to find income producing tracts in Texas for a decent price.