We took an 11 month old to a game last year because of plans and circumstances changing last second. While it's not something I would recommend, it wasn't that bad.
He enjoyed March In, the intro with fireworks and music, and the noise. He enjoyed the band at halftime and actually napped on my lap through the entire 3rd quarter. (So if the question ever gets asked how boring the 2015 offense was, the answer is enough to put a baby to sleep.) The people around him laughed, talked, and waved to him, and being that it was Kyler Murray at QB and Western Carolina as the opponent, we were restless and ready to leave long before him.
I don't expect these memories to stay with him any more than a 1st birthday or celebrating his first Christmas or Halloween that year, so people using that reasoning seems a little silly. I'm guessing the people answering negatively either don't have a lot of experience with kids, or the ones they do have experience with were little ****s.