The CEO of the X Games and the former head of Google’s AI Solutions Architecture are teaming up to launch The Owl AI, a set of artificial intelligence tools designed to improve sports judging, refereeing, talent evaluation, coaching, and athletic performance. Along the way, The Owl AI also can deliver live, on-screen commentary in 40 languages other than English, enormously expanding the possible reach of televised experiences.

The company today announced it has raised $11 million in a seed round led by S32, the venture fund founded by Google Ventures creator Bill Maris, with participation from Menlo Ventures and Susa Ventures. Josh Gwyther, who just left his job as head of Google’s A.I. Solutions Architecture after nine years with the company, is CEO.

(Logo courtesy of The Owl AI)

The company’s executive chairman is Jeremy Bloom (no relation), who has headed the X Games since late last year and formerly was a World Cup skier and pro football player.

“We can judge games, and can referee purely objectively, not subjectively,” said Bloom. “We want to contribute to having a solution there, not to replace referees and judges but to assist them in making the right calls.”

The Owl technology got its first test run at the Winter X Games in Aspen last winter, helping judges evaluate the fast-moving and complex tricks of the snowboarding SuperPipe competition. It delivered on-screen commentary on who the competition’s likely winners would be, based on their practice runs and history (see video at bottom). The tech will be used again at the X Games summer gathering this weekend in Salt Lake City, as part of the men’s and women’s skate park competitions.

“Robo” judging has been creeping into many sports in the past few years. Video-assist refereeing helps soccer officials review offsides, hand balls, fouls, and similar high-consequence issues. Major League Baseball has been experimenting with robo umpires to call strikes and balls in minor league games. And major tennis tournaments have used video playback to review whether a shot is in bounds or not.

Separately, former Microsoft CEO Steve Ballmer leveraged his ownership of the NBA’s Los Angeles Clippers to launch Second Spectrum, a tech company later acquired for $200 million by Genius Sports that provides data tracking and visualization of athletes.

“(The Owl AI) is different in a lot of ways because there’s an intelligence behind it,” Gwyther said. “Most (existing referee-assist systems) were built on a vision model. It’s just based on pure ball-in, ball-out (judgments). It’s binary. We can do things that a traditional technology couldn’t, like judging subjective sports: (athletes’) style, flow, how they land, how they perform.”

That means, in the snowboard and skateboard sports in X Games competitions, understanding what creates good style and form on a board, Bloom said.

“Style is good economies of motion in air,” Bloom said. “When you look at who wins or loses, it’s the person who does the most difficult trick, who lands the cleanest, and is on target.”

Other questions need to be answered in a “tricky,” trick-filled sport such as skateboarding, Bloom said.

When the athlete grabs the board in a trick, do they just touch the board or actually grab it? Do they lift an arm to right their balance, suggesting they’re off perfect alignment? How many spins of the board or themselves did they actually complete? How cleanly did they grind a rail? How did they land the board?

“The easiest way to think about it is the tech allows us to put a Ph.D.-level analyst in the game, and create structured analysis behind it,” Gwyther said. “You can do analysis on every player in the game, tuned for that position, that player. That’s the exciting part for me.”

Unsurprisingly, the system’s “core technology” is built atop Gemini, the Alphabet-owned Google AI initiative on which Gwyther formerly worked.

“We will be doing adaptive tuning on top of that,” to optimize the tools for different sports, said Gwyther.

Computer vision and other AI technologies have advanced rapidly in recent years, but only in the past six months were they sufficiently sophisticated to handle what The Owl is doing, Gwyther said, bringing enough nuance to pick up performance details at ever-higher video frame rates and resolutions.

“At this point, you can feed it everything from 720P (resolution) to 4K, it doesn’t really matter,” Gwyther said. “Not only can it interpret that video, it can be produced on a cell phone.”

And out of that, many other uses become possible, Gwyther said.

“Once it becomes a really good judge, it can be a really good coach,” Gwyther said. “You can do analysis” to evaluate talent for, say, college scholarship offers, or pro sports drafts.

It also can help improve an athlete’s form, to improve performance. High-end sports-motion clinics have emerged in recent years to evaluate top athletes and help optimize their skills and form, though the cost of such analyses typically have been well beyond the wallets of most aspiring young athletes.

A third component leverages the AI’s understanding of what’s happening onscreen to generate live commentary in any of 40 languages, enormously expanding potential fan engagement beyond the native language of any given sportscast.

MORE FOR YOU

Bloom acknowledged that the technology also likely will be attractive to many in the exploding business of sports gambling, worth an estimated $16 billion a year now in the United States, as they try to arbitrage the best available bets in a given sport, especially compared to what may be posted by gambling sites. Even in the Aspen X Games test run, the tool predicted a top 3 order of finish that differed from the betting lines set by oddsmakers.

“I think this kind of technology will be ubiquitous and table stakes for Vegas to set odds, but also for picking your fantasy team,” Bloom said. Also, “how do you find the best odds?”

Given the rapid expansion of sports betting, getting refereeing and judging consistently, objectively right will become ever more important for leagues to rebut any possible complaints of improper influence, Bloom said, while also doing anomaly detection.

“If you’re (NFL Commissioner) Roger Goodell and you have 16 games every weekend, and the technology flags, ‘Hey, this ref’s behavior is a bit off, and they missed five calls,’” then the league can quickly and directly follow up to see what’s going on, Bloom said.

After the Winter Games debut, “six or seven” other leagues reached out about the Owl technology and how it might supplement their referees/judges’ work, Bloom said, though he said no possible deal is ready for announcement yet.

Watch the video below from the Winter X Games to hear The Owl AI predict he top three finishers in the SuperPipe: