A rivalry is basically a fun sort of feud. And is it ever fun!
We rarely see them outside of sports games, because a necessary condition is that victories and defeats are not final. A nemesis or archenemy is different from a rival in that a rival cannot be permanently defeated. Each time you defeat a rival, there's a chance that rival will be back one day to defeat you. Victories are only temporary.
In fact, for the rivalry to retain emotional gravity, the competitive balance must remain relatively even. If you know you're always going to win against the enemy, then there's little emotional investment in the game.
Another necessary element of rivalries is history. Pointing a player to another as his rival can add a bit of fun, but the emotional investment in that rivalry is meek compared to if the two players shared a history. The longer and more eventful the history, the more powerful that rivalry becomes... which is why I can't wait to see Bama roll over Auburn this year!
Mario vs Bowser can be called a rivalry in Mario Kart and Super Smash Brothers. I grew up on Nintendo consoles, so I knew those two had been locked in a never-ending battle for years. I still would not call that rivalry if the those characters met only in Mario adventure games, but the many competitive spin-offs have allowed Bowser to have his share of wins, too.
Even so, Bowser and Mario share a public history only (to my knowledge) in the adventure games. Their interactions in the competitive spin-offs are limited to personal histories. Mario always wins in the adventure games, no matter who's playing. Bowser's victories were seldom recorded and shared.
Notice that those Mario spin-offs in which a rivalry exists are, again, sports games.
Warhammer Online is a non-sport that includes rival factions. For long-time Warhammer IP fans, those rivalries have long and eventful histories, so they are powerful. But for players new to the IP, like me, WAR is like suddenly joining a centuries-old feud you know nothing about. Thankfully, the history there is a living history, seen in the game's environments and felt throughout gameplay. But the rivalries are still shallow in comparison to sports if you're new to the IP. And while sports have dozens of teams with many rivalries, WAR is limited to just three (not counting the broad Order vs Destruction).
So, considering all that, how can deep and lasting rivalries be included in non-sports games or developed over time among their players? How can that gap between sports and non-sports be better bridged?