This sounds like a worryingly bad idea on many levels.
1. "Such a system could find, fix and finish a target without external direction by a human controller."
The fact that human-out-of-the-loop kill decisions is a bad idea should not require further elaboration.
2. "Suddenly, a simulated threat radar activated. The pair of networked UCAVs immediately classified the threat and executed a plan to destroy it based on the position of each X-45 in relation to the target, what weapons were available, what each drone’s fuel load was and the nature of the target itself. The calculations happened in a blink of an eye."
Congratulations, now all of the adversarial techniques that make state-of-the-art neural networks fall over will apply to fighter jets as well. Does the system deterministically plan to attack everything it sees that looks like a threat? What about easy threats that show up first, followed by progressively more challenging / fuel-draining threats? Now instead of radar jammers, the signals guys will go back to building elaborate decoy generators.
Without a human in the loop, our current autonomous technologies are woefully inadequate for determining whether or not something smells like a trap. Our enemies are going to learn how to knock these things out of the sky with a $200 SDR and a power amp.
Not sure if the article was edited before you posted, but the author covers this. There is a human in the loop. The swarm sends all its data to the ground station, where a human gives go/no-go.
There thing that is automated is the first phase threat identification and the generation and execution of the attack plan, not the threat verification or decision to execute. Which is fine IMO.
That's always in the narrative to make you feel better. If the military gave a shit decisions about attack plans, we wouldn't have missiles.
With a plane-sized stealth drone, you can use inertia/gyro guidance to eliminate the GPS risk. Seems silly to build a stealth aircraft that has to transmit and receive instructions. Where do vehicles like this fit? You have long-range cruise missiles, cheap, long loitering drones with hellfire missiles, manned aircraft, and these unmanned fighter-jet class devices already.
A weapon like this wouldn't have super-long loitering time, but would be potentially more stealthy and more disposable than a manned aircraft. It's pretty obvious why it's secret... it's an air-defense buster, strike platform against a nation-state (vs. rabble), delivery truck for special bombs, etc or a nuclear weapon delivery truck.
> It's pretty obvious why it's secret... it's an air-defense buster, strike platform against a nation-state (vs. rabble), delivery truck for special bombs, etc or a nuclear weapon delivery truck.
> Without a human in the loop, our current autonomous technologies are woefully inadequate for determining whether or not something smells like a trap. Our enemies are going to learn how to knock these things out of the sky with a $200 SDR and a power amp.
If AI can beat humans at strategy & bluffing games like Starcraft & poker, why couldn't they defeat humans at detecting & defeating anti air traps ?
Those are games with rules and mechanics that are well-specified. A real-world battlefield environment is not nearly as well-specified. Sensors are noisy and can be tricked in all sorts of silly ways.
Also the goal isn't necessarily to beat the other player -- making their planning system crash or lose confidence in itself can be more than adequate.
You're forgetting it's not actually AI. They'll perform well at whatever task we adequately train them to do. It's an arms race - we train them to defeat anti air traps, the adversary will move the goalpost somewhere else.
Can't help but recall a relative episode of StarTrek in which an AI computer takes control of the Enterprise and starts targeting friendlies as it classifies them as hostile.
This actually reminds me of an episode of Start Trek: Voyager (Dreadnaught).
A missile with an advanced AI gets transported to the Delta quadrant in the same way as Voyager was. It was already en-route to a target planet. Obviously it lost lock. It then used what information it had about the planet to find it again. It found it – except it wasn't the same planet, it was a completely innocent one. And set a course. It correctly calculated that the probability of two planets having the exact same parameters was infinitesimally small, so it must be the correct target. It was further unconvinced that it was, in fact, in the Delta quadrant.
It also disregarded the orders to shut down, which came from the same engineer who reprogrammed it, believing that she was a rogue agent.
Ah yes, I remember that one, more you think about it - this lesson has been played out many times in various scifi forms. Even to the extreme of Terminator and skynet.
Makes you wonder what will play out in real life as we go into the AI age.
Or if it has already played out many times in history as people operating on incomplete information have made bad decisions and then stuck with them even in the face of newly incoming conflicting data.
Really the worry here is that AIs will be just as bad at decision making as humans.
A semi-recent example is the shooting down of MH17, which was shot down by Russians or their allies because they mistook an airliner for a military aircraft, even when their flight profiles were totally different. The controllers were expecting a military aircraft to arrive around that time and didn't stop to consider the alternatives, even when the target didn't behave properly.
Yes the classic 60's saying from computers "Garbage In Garbage Out" still holds true. Though today we can process that garbage faster and react faster - what could go wrong.
> human-out-of-the-loop kill decisions is a bad idea
Maybe so, but this is an inevitable arms race. What happens when the other party has drones and we ned to respond faster than the human can make the decision?
> now all of the adversarial techniques that make state-of-the-art neural networks fall over will apply to fighter jets as well. ... Without a human in the loop , our current autonomous technologies are woefully inadequate for determining whether or not something smells like a trap.
I'm not sure humans help here. The interesting adversarial examples today are complex images that humans are uniquely good at parsing. But these things are doing to be consuming radar, signals, etc that humans don't really have an advantage with. Not saying decoys etc won't happen, but I don't think humans help here.
This is a fascinating article that is well written and has a lot of interesting information.
However it feels like he is falling into the same trap that most of the media falls into constantly, thinking that AI & Software are easy.
If he thinks the F-35 was hard to launch with all it's software that's nothing compared to an autonomous combat swarm with all the features he's talking about. You can't rush out stuff that controls machines killing stuff.
My guess is he's more of an aviation guy than a software guy so this stuff somehow doesn't seem as hard as it is.
He also mentions the USAF has had a lot of black programs but then continues on with the thesis that they're ignoring this as opposed to continuing to develop in the black.
Also these swarms will be least effective in the kinds of operations we've been mired in. It's a lot easier to set a swarm in motion with orders to clear the sky in a WW3 scenario than it is to have them try and shoot a particular pickup truck filled with civilian-clothed possible enemy combatants in an urban area in Afghanistan.
As long as you've observed the particular pickup truck before IDing it later is actually a very solved problem (like, the exact one, not just make and model). Back when I was an intern I worked on the "old boring currently deployed legacy tech" and it was quite good (probably scary good to some people) and it was using radar signatures. I suspect that the development of that technology combined with recent visual spectrum developments (i.e. cheap good cameras) has made the tech even better.
Having the good judgement to not obliterate the truck while it's yielding for a funeral procession is the hard part to do in software.
I agree he's underestimating the software complexity. Anyone can built that crap. It takes tons of R&D man hours to refine it into something that actually works and works consistently in the real world.
I think he is talking about decision making abilities. But even stuff you mentioned are too hard to build for a real world military scenario. They have to deal with countless different objects, scenarios, and situations. Too many edge cases to cover and will take ages to adjust and it will be full of human errors.
They can build it for simple tasks. Find the best route to go there, bomb that specific target, come back and during your mission deal with a limited set of defense scenarios.
14) The USAF has discovered that there are insurmountable problems that have prevented the development of effective autonomous systems. Therefore they have declined to invest in technology that isn't working.
I would suggest that manned attack craft are as important to the Air Force as the Carrier is to the Navy. Neither will let go until circumstances outside of their control over Congress forces it.
The AF probably realized that in a contested environment, drones would be useless. If the Iranians can shoot down a $200M Global Hawk with homebrewed missiles, the Russians and Chinese will have no problem. And if you can't shoot them down, just jam GPS etc. That's how the Iranians captured an RQ-170.
Wow, was just reading this article after reading the one about SAPs. I don't have much of any knowledge on aerospace engineering or what the Air Force/Navy have planned for our skies, but it's still a crazy thought that they'd abandon that kind of tech. I understand the upper echelons of governments get really bureaucratic but a lot of the times their decisions are incredibly simplistic, a program like this could be abandoned just because the top rung shift focus to intelligence gathering, any platform that doesn't fit that purpose or can easily be made to fit that purpose gets the axe, despite how useful it may be in the future. All boils down to money eventually.
I'm really interested in what the "next phase" of aviation combat will be, or if there even will be a next phase. It's possible some organization would create a weapon that would physically disrupt any vehicle or object flying through a certain patch of sky - that may push us into a air-to-groud laser or smart-guided weapon age to counter those kinds of measures, and also means air vehicles will need serious intel platforms to get around them. We also may see vehicles that can transition from air to sea and retain full combative power, anything in the skies that can't float at that point goes the way of the do-do, no matter how intelligent it's systems are, so maybe the USAF is waiting until it's clear what will work for the next 50 years, while other branches invest UCAV technology.
Though I'd much prefer to have advances in the technology of diplomacy than that of war, I think having a number of semi-autonomous small drones in formation flight around a manned aircraft makes more sense than AI drones. The drone swarm could detect the attacks you describe destructively if needed, so the pilot can dodge.
How many stealthy UCAVs are flying publicly? As far as I'm aware the answer is still 0. (Well, 1 if you count the XQ-58 Valkyrie [0], which has flown twice)
I was under the impression that there are a few stealth UAVs being operated currently.The Sentinel comes to mind and I think both China and Russia have working prototypes.
The article speaks to the RQ-170, introduced 9 years before the article and is one of your links. The idea is that the Air Force abruptly pulled out of a join effort with the Navy and didn’t seem to have an alternative. What happened? Does the AF have something else that is not public or did the manned aircraft program see UCAV as threat to eliminate?
The RQ-170 is a stealth UAV, but not a UCAV - it's unarmed. The other two aren't US, and so are more 'worrying' than 'reassuring' when it comes to figuring out where the US's stealth UCAVs are.
And you are not going to tell us about it or give us any links to read. But you think we still should believe you.
All that makes me think that things actually have not changed, but you don't want to admit it, and so you are claiming they have changed. But perhaps I am mistaken about that.
The USAF is making the right call as far as I'm concerned. It is breathtakingly easy for hostile actors to manipulate the behavior of UAVs. Hell, Iran stole one by just spoofing its GPS and landing it exactly where they wanted. Giving every hostile nation and hacker the opportunity to find zero-days on a fleet of supersonic aircraft is stupidly reckless.
I get the impression that the author vastly overestimates the capabilities of modern software. We don't even have reliable self-driving cars, yet they are hoping for reliable self-flying planes that can decide when and how to kill? Even the task of collecting training data for such a system would be monumental.
The B-2s problem is that it was obsolete before it was even built. Long range strategic stealth bombing isn't really a useful mission profile in a world with ICBMs or even ship launched cruise missiles.
Yeah, but you need to add in the cost of the B2 per flight hour on top of that. It's hard to argue a cost savings with anything related to the B2 program.
>While the future looked blindingly bright for UCAVs, against all logic, the great UCAV revolution would be buried without explanation, and with it the promise of an operational and devestating unmanned swarm.
Oh look, another lovely article from the Global Non-Denominational Death Cult.
Well there are clearly multiple denominations of this death cult. How else would each denomination justify their funding, if not to prevent the wrong denomination from making offerings.
Yes, but between them they have set up a global non-denominational cooperative for organising multi-denominational meetups and activities, such as the biannual sacrificial beach party.
1. "Such a system could find, fix and finish a target without external direction by a human controller."
The fact that human-out-of-the-loop kill decisions is a bad idea should not require further elaboration.
2. "Suddenly, a simulated threat radar activated. The pair of networked UCAVs immediately classified the threat and executed a plan to destroy it based on the position of each X-45 in relation to the target, what weapons were available, what each drone’s fuel load was and the nature of the target itself. The calculations happened in a blink of an eye."
Congratulations, now all of the adversarial techniques that make state-of-the-art neural networks fall over will apply to fighter jets as well. Does the system deterministically plan to attack everything it sees that looks like a threat? What about easy threats that show up first, followed by progressively more challenging / fuel-draining threats? Now instead of radar jammers, the signals guys will go back to building elaborate decoy generators.
Without a human in the loop, our current autonomous technologies are woefully inadequate for determining whether or not something smells like a trap. Our enemies are going to learn how to knock these things out of the sky with a $200 SDR and a power amp.
There thing that is automated is the first phase threat identification and the generation and execution of the attack plan, not the threat verification or decision to execute. Which is fine IMO.
With a plane-sized stealth drone, you can use inertia/gyro guidance to eliminate the GPS risk. Seems silly to build a stealth aircraft that has to transmit and receive instructions. Where do vehicles like this fit? You have long-range cruise missiles, cheap, long loitering drones with hellfire missiles, manned aircraft, and these unmanned fighter-jet class devices already.
A weapon like this wouldn't have super-long loitering time, but would be potentially more stealthy and more disposable than a manned aircraft. It's pretty obvious why it's secret... it's an air-defense buster, strike platform against a nation-state (vs. rabble), delivery truck for special bombs, etc or a nuclear weapon delivery truck.
This
If AI can beat humans at strategy & bluffing games like Starcraft & poker, why couldn't they defeat humans at detecting & defeating anti air traps ?
Also the goal isn't necessarily to beat the other player -- making their planning system crash or lose confidence in itself can be more than adequate.
I don't think these problems are comparable.
https://en.wikipedia.org/wiki/The_Ultimate_Computer
A missile with an advanced AI gets transported to the Delta quadrant in the same way as Voyager was. It was already en-route to a target planet. Obviously it lost lock. It then used what information it had about the planet to find it again. It found it – except it wasn't the same planet, it was a completely innocent one. And set a course. It correctly calculated that the probability of two planets having the exact same parameters was infinitesimally small, so it must be the correct target. It was further unconvinced that it was, in fact, in the Delta quadrant.
It also disregarded the orders to shut down, which came from the same engineer who reprogrammed it, believing that she was a rogue agent.
https://memory-alpha.fandom.com/wiki/Dreadnought_(episode)
Makes you wonder what will play out in real life as we go into the AI age.
Really the worry here is that AIs will be just as bad at decision making as humans.
A semi-recent example is the shooting down of MH17, which was shot down by Russians or their allies because they mistook an airliner for a military aircraft, even when their flight profiles were totally different. The controllers were expecting a military aircraft to arrive around that time and didn't stop to consider the alternatives, even when the target didn't behave properly.
Maybe so, but this is an inevitable arms race. What happens when the other party has drones and we ned to respond faster than the human can make the decision?
> now all of the adversarial techniques that make state-of-the-art neural networks fall over will apply to fighter jets as well. ... Without a human in the loop , our current autonomous technologies are woefully inadequate for determining whether or not something smells like a trap.
I'm not sure humans help here. The interesting adversarial examples today are complex images that humans are uniquely good at parsing. But these things are doing to be consuming radar, signals, etc that humans don't really have an advantage with. Not saying decoys etc won't happen, but I don't think humans help here.
However it feels like he is falling into the same trap that most of the media falls into constantly, thinking that AI & Software are easy.
If he thinks the F-35 was hard to launch with all it's software that's nothing compared to an autonomous combat swarm with all the features he's talking about. You can't rush out stuff that controls machines killing stuff.
My guess is he's more of an aviation guy than a software guy so this stuff somehow doesn't seem as hard as it is.
He also mentions the USAF has had a lot of black programs but then continues on with the thesis that they're ignoring this as opposed to continuing to develop in the black.
Also these swarms will be least effective in the kinds of operations we've been mired in. It's a lot easier to set a swarm in motion with orders to clear the sky in a WW3 scenario than it is to have them try and shoot a particular pickup truck filled with civilian-clothed possible enemy combatants in an urban area in Afghanistan.
Having the good judgement to not obliterate the truck while it's yielding for a funeral procession is the hard part to do in software.
I agree he's underestimating the software complexity. Anyone can built that crap. It takes tons of R&D man hours to refine it into something that actually works and works consistently in the real world.
They can build it for simple tasks. Find the best route to go there, bomb that specific target, come back and during your mission deal with a limited set of defense scenarios.
I'm really interested in what the "next phase" of aviation combat will be, or if there even will be a next phase. It's possible some organization would create a weapon that would physically disrupt any vehicle or object flying through a certain patch of sky - that may push us into a air-to-groud laser or smart-guided weapon age to counter those kinds of measures, and also means air vehicles will need serious intel platforms to get around them. We also may see vehicles that can transition from air to sea and retain full combative power, anything in the skies that can't float at that point goes the way of the do-do, no matter how intelligent it's systems are, so maybe the USAF is waiting until it's clear what will work for the next 50 years, while other branches invest UCAV technology.
*air-to-ground, not ground-to-air
Remember the APC Bradley
"Boeing’s control software ... was called DICE, or Decision Mission-Control Software."
Okay, well, I see where the D and the C in the acronym comes from, but...
0: https://en.wikipedia.org/wiki/Kratos_XQ-58_Valkyrie
https://en.wikipedia.org/wiki/Lockheed_Martin_RQ-170_Sentine...
https://en.wikipedia.org/wiki/Sukhoi_Okhotnik
https://en.wikipedia.org/wiki/CASC_Rainbow#CH-7
All that makes me think that things actually have not changed, but you don't want to admit it, and so you are claiming they have changed. But perhaps I am mistaken about that.
For the cost of about one of those cruise missiles ($25k/JDAM versus $1.4m Tomahawks), and a lot less likely to be intercepted by air defenses.
https://combataircraft.keypublishing.com/2019/02/27/f-117-ni...
Oh look, another lovely article from the Global Non-Denominational Death Cult.