A couple of observations from watching parts of three days of this @DARPA competition, where AI algorithms serving as fighter pilots in simulated dogfights squared off. (thread) https://twitter.com/ZachFB/status/1296837726067728385

First off, the AI system that won was definitely engaging in the kind of reward hacking behavior we see when algorithms try to find an edge in games. It was basically committing to kamikaze runs, charging its opponent to within 100 ft while locked in and firing.

This is mentioned in the story, but that kind of flight behavior would be really risky for a real world pilot, as he or she could fly right through a debris field created by their slain foe. It's also against the rules in any normal dogfighting contest.

But since it wasn't off limits for this competition, the AI found the weak point and exploited it expertly. There were other areas where the AI systems really pushed the limit on what the model F-16 could really do.

The AI aircraft pretty consistently skirted the edge of stalling the aircraft (I saw one drop to about 100 knots during a maneuver). If the aircraft stalls you run the risk of a complete loss of control and a crash, so human pilots typically would be hesitant to push this hard.

That said, it's an example of where AI could be handy, as a computer can come much closer to the performance envelope of a system without tipping into loss of control.

I also saw the AI aircraft holding impossibly long sharp turns. One of the limits in normal dogfighting is that pilots have to keep blood in their brains while g forces try to pool it in their feet. That means you can't just sit in an aggressive bank indefinitely.

The AI aircraft would hold 7-8 g turns for minutes at a time. I'm not a pilot but I've always heard about 6 is what you can do consistently. The systems were limited so that they were not supposed to sit over 9 g's, although I did see one flash past 11 at one point.

All of that said, the winning system from Heron did show one superhuman skill that could be very useful - it's ability to keep perfect aim on a target. Once it locked on its gun was absolutely perfect, which gave it an edge in the head on jousting rounds it favored.

Knowing how good the Heron AI was at this tactic, the human F-16 pilot who competed at the end clearly focused on trying to avoid being rapidly gunned down by a suicidal assault from the AI. He was able to maneuver away from these runs, but he never got into an attack position.

There was one major impediment that the human pilot (Banger) had that might have really shaped the contest - Banger was using VR goggles and had to actually keep visual contact with his opponent. The AI had perfect physical awareness, so it always knew where Banger was.

That's a big deal in dogfighting, as pilots have to struggle while confronting g's (although to be fair Banger was just sitting in a chair on earth for the sim) to keep eyes on their opponent. This computer system didn't.

Anyway Banger was in a tough position facing off against an AI that had zero regard for self-preservation, and always knew where he was. That doesn't take away from the AI itself, which was impressive. It found a tactic and largely executed it perfectly.

The AI also wasn't only about this one charging tactic even if it was its primary approach. When Banger got out from the initial charges it consistently managed to stay after him and win.

So in conclusion, the AI did what an AI does - it found a way to game the system, and then used its precision to ruthlessly win. An impressive technical feat but there's still a ways to go before you'd actually want to hand it the keys to your jet. (End)

Oh, and if you missed the link to the actual story up top, and traveled this far in the thread, you should go read it. https://publicintegrity.org/national-security/future-of-warfare/machine-beats-man-in-air-combat-simulations/ It's what actually pays the bills so that I can do this reporting.