I seriously doubt it was the early stages of dementia. That's the first I've ever remotely heard that, and I usually keep up a lot with info over the years on Coach Royal. The fact that he lived to be the age he did probably indicated anything but that early of a striking of that disease. I'm not saying it's impossible, just very unlikely. I met him in the late 1990's after a spring game, and it was readily apparent he was still extremely sharp and alert. That would have been nearly a quarter century after retiring, so again it doesn't line up.
Regarding his departure, yes there were some enemies, but as big a factor as any (perhaps the most critical) is that he could sense clearly that the losses were so much more painful than the joy of the victories, that it was no longer worth it. No doubt much of that was fueled by the bitter rivalry with the blatantly cheating Barry Switzer, who Royal challenged to take lie detector tests with (as well as each team's whole football staff). Royal knew Switzer's group had been spying on practices, not to mention all the underhanded recruiting going on. Barry was apparently not above just about anything to win in the recruits' living rooms and on the field. Royal, as fiercely competitive as anyone, resented even having to play what he referred to as "that monster program."
Royal got whipped a few times by a frankly superior Switzer club, but several of those contests were enormously competitive despite the Sooners have probably the most talented team in the country year in and year out in that period. Even one of the whippings was greatly deceiving, because a dominant OU team led Texas only 3-0 late in the game before blocking, of all things, a quick kick. Royal hadn't used the QK since the early 60s, when such a conservative move was far more common. Yet, when Texas lined up prior to the QK, the Sooner players started screaming "Quick Kick, Quick Kick!" That is where the spying accusation/situation comes into play. OU knew what kind of adjustment in our lineup to look for due to spying, and they took full advantage, blocking it for a touchdown. The floodgates, naturally, opened after we'd held that powerhouse to three points all game (they had probably been averaging about "half a hundred" per game going in (a phrase Switzer liked to use referring to dominating most of his opponents by scoring 50+)).
Anyway, the whole thing became too much for Royal, who actually dry heaved following the 6-6 tie in his last year vs OU in '76. Texas had played masterful defense all game and, Earl-less, led 6-0 late in the contest. Unfortunately, a back named Ivey Suber fumbled, allowing OU a short field for what gave them the tying TD. Mercifully, the Sooners botched the extra point.
In summation, despite those late year struggles with $witzer's $ooners (sorry, couldn't resist), DKR ended his 20 glittering seasons at Texas without ever having a losing season, winning 11 (I believe) conference titles and at least a share of three national ones, along with several other close calls. Many believe two of his teams that didn't win national titles were possibly his most powerful, which were the 1961 and 1968 squads. It should be noted each of those two teams as well as the three national title teams were split into different eras and different offenses. Up thru '63-64, players handled offense and defense. By '65, the switch to full-fledged specialization had been made, and it took Royal a few years to adjust, along with a rash of injuries and the general failings of an expected "can't miss" QB in Bill Bradley (though to be fair, he tore up his knee one of those years).
Bradley, despite changing positions to DB (where he became a perennial all-Pro in the NFL), contributed greatly still, as Texas roared to a dominant finish in '68, a season that marked the arrival of DKR's Wishbone, which schools across the country emulated.