In early 2024, the first videos of the use of Ukrainian FPV drones with machine vision technology appeared - the drone "captured" a target and destroyed it on its own, despite losing contact with the pilot.
This has everyone talking about another revolution in FPV technology. The popular idea that machine vision will allow you to easily bypass enemy electronic warfare and make drones many times more effective has become popular.
Fantasy also painted more futuristic scenarios - that the appearance of machine vision on combat FPV drones is a direct ticket to swarms of drones that will be able to go on a "hunt" on their own, search for enemy equipment and quickly destroy it on command.
But a year has passed, and there is still no sign of widespread use of drones with machine vision. Instead, the military, government, and manufacturers are investing much more resources in a more reliable means of protecting communications — fiber-optic drones.
Journalists of the Oboronka project Dmytro Kazantsev and Bohdan Miroshnychenko spoke with Ukrainian developers of guidance technologies, drone manufacturers, military personnel, and state representatives to understand when FPV with machine vision will become a truly mass phenomenon on the front.
What is machine vision and how does it work?
The idea of teaching a drone to see an enemy target and "crash" into it on its own in the last tens of meters without the operator's participation did not come out of nowhere - it was caused by two key factors.
The first is the emergence of an increasing number of enemy electronic warfare (EW) and electronic reconnaissance (ERR) systems that analyze the operating frequencies of Ukrainian drones and jam their communications in the last meters to the target.
The second is that when operating at long distances, the terrain and limited radio horizon make it difficult to hit the target, because closer to the ground, communication with the drone is lost. Therefore, there is a need for software that will independently guide the drone to the target.
It should work like this: an FPV drone, equipped with an automatic pattern recognition algorithm, after being “captured” by the operator, memorizes the outlines of the target and navigates to them during the flight. The drone keeps the image of the target, tracks its movement and flies to it on its own, even after complete loss of contact with the pilot.
The emergence of high-quality machine vision technology and its skillful use will mean complete helplessness of enemy electronic warfare in the last hundreds of meters to the target, because the command to destroy is given outside its zone of action. But this is only in theory. In practice, developing high-quality software that will actually work in combat conditions is not an easy task.
The complexity of this technology was explained to Oboronka journalists by the VGI-9 company, which develops software for Ukrainian combat FPVs. The target guidance process takes place in several stages.
First — capturing visual data from the camera. Then — image processing: filtering, enhancement, contour selection. Next comes object, pattern, and motion recognition using a neural network. And finally — autonomous decision-making in drone control based on image analysis.
And all this must happen in a matter of seconds, at high speed, with surgical precision, and also take into account various external factors such as drone load, wind gusts, target speed, etc.
The declared functionality of modern Ukrainian homing modules is broader than it might seem at first glance. For example, the FPV drone “General Chereshnya” is equipped with the VGI-9 module from the “Engineering Corps”. It has built-in cruise control, which allows the drone to fly to coordinates without the participation of a pilot, zoom of the main and additional screens, the function of correcting the flight trajectory when the target is already captured, quick transition to manual control, the ability to work at dusk, built-in wind correction algorithms, the ability to detect camouflaged targets, etc.
Ukrainian models of FPV drones with machine vision perform excellently on the training ground, but demonstrate mixed results on the battlefield.
Is machine vision working on the front lines?
"The guidance works normally on kamikaze aircraft that fly long distances. It is very necessary there, because due to the refraction of the radio horizon, video communication is lost on approach to the ground.
"On quadcopters, this is more difficult, because it is often a problem for the military to even bring the drone to the distance of "capturing" an enemy object. In addition, the homing does not cope well with moving targets. Another problem is FPV cameras, which are of insufficient quality to recognize targets at distances of 500 meters. Many have machine vision, it works "somehow", but it has not become a technology that changes the course of war," the interlocutor noted.
Reviews from the military about Ukrainian drones with machine vision can be mixed.
One of the Ukrainian UAV operators, who wished to remain anonymous, told Oboronka that machine vision helps in certain combat situations, but remains weak as a means against electronic warfare.
"In my opinion, if there is electronic warfare suppression, you won't even be able to fly to the target. But the technology is suitable for bypassing terrain obstacles and in conditions where the drone goes beyond the radio horizon. This is also very important, because nature does not always provide ideal conditions for work. We work with additional guidance, because it is better than without it. Even if it does not hit the target exactly, without it the chances of hitting would be much lower," the military man said.
Going beyond the radio horizon refers to a situation where communication with the drone is lost due to the “rounding” of the Earth. And terrain obstructions are when the terrain itself prevents radio waves from reaching the drone in a straight line.
Igor Raikov, a platoon commander in the anti-tank company of the 13th brigade of the Charter National Guard, told Oboronka that the attention of their operators is currently focused on mastering the use of drones on fiber optics and conventional radio communications, because it is for these types of drones that there is the most work at the front.
Read also: How Ukraine is deploying drone production "on fiber optics"
“The Russians have learned to hide and camouflage well, so only a living person can identify and “capture” such a target,” said Raikov, adding that currently drones with machine vision do not look very similar to what they should ideally look like.
The Brave1 state defense cluster told Oboronka journalists that FPV drones with machine vision-based guidance are a solution that is already working on the front lines and is being purchased by the state. But the technology still has certain drawbacks:
▪️insufficient camera quality, which prevents machine vision from correctly capturing objects;
▪️the operator's inability to direct the drone to a specific point, for example, a vulnerable spot on a tank, which may reduce the effectiveness of use;
▪️difficulty in re-equipping already-made drones with new modules.
"Developing an automatic guidance system for FPV drones is a challenge at the intersection of several complex disciplines: machine vision, tracking algorithms, optimization for limited computing resources, and operation in unstable combat conditions. On the Ukrainian market, more than 40 teams are developing their own guidance systems. However, the problem is that many teams, having a still "raw" solution, are in a hurry to present it to the military, purchasers, or manufacturers.
As a result, they demonstrate a product that is not yet ready for combat use - and this creates general skepticism about the entire market of machine vision systems. The military sees unsuccessful attempts and concludes: "the technology does not work." We are consciously working to break this myth. The optical guidance system is not the future, it is a tool of today that really works, destroys the enemy and helps the military save lives," said the developers from the VGI-9 company.
The problem of mass and effective implementation of machine vision technology may also be related to the lack of unification - there are hundreds of drone manufacturers in Ukraine that use different components.
According to Oleksandr Soloshchenko, chief engineer of the company General Chereshnya, which produces FPV drones, machine vision systems are not universal and are integrated only for specific hardware. For example, for certain flight controllers, which are not always available.
"If we take other components for the drone, then we need to integrate the software product there again. This takes some time and resources," the engineer explained.
The need for unification in the FPV drone segment has been long overdue. This is a matter for government procurement agencies, which should clearly define component standards and make drones more or less identical in terms of basic characteristics, rather than further encouraging a "zoo" of technologies.
"Another problem is that working with guidance requires additional skills from the pilot. Not all pilots have them and need additional training. Our partners have created a simulator that allows even an inexperienced pilot to learn how to fly a drone with guidance," Soloshchenko added.
It is obvious that the focus of manufacturers and the military is now on technologically simpler drones with fiber-optic control. In essence, they solve the same problems as drones with machine vision - operation under the influence of electronic warfare and in difficult terrain.
But according to Oleksiy Babenko, director of Vyriy, one of the largest manufacturers of FPV drones, fiber optic communication can conceptually complement, but not replace, drones with guidance.
"With fiber-optic drones, we can cover short distances: 10, 15, even 20 kilometers. With guidance systems, it's a different matter, it's about long-distance flights," Babenko said in an interview with Militarnyi.
Should we expect a new technological breakthrough from machine vision?
"We will not name specific dates for the appearance of a ready-to-use solution yet. However, we know that there are already teams in Ukraine that are actively testing a "drone swarm". Some of them declare plans to provide a ready-made solution in the next 3 months. In our opinion, such an assessment seems too optimistic. It is quite likely that in a year or a half we will see examples of the combat use of a drone swarm. What this swarm will be like and under what combat, weather or other conditions it will be used is another question," said the engineers and developers of the VGI-9 team.
"The technology has proven itself well. The trend is that the use of drones with guidance/machine vision modules will become more and more common. Teams of engineers are constantly working on improving these systems, which will make them even more effective in the future," said Oleksandr Soloshchenko.
Brave1 said that currently there are 100+ companies involved in the production of drones with machine vision in Ukraine.
The integration of auto-guidance is a completely logical technological step in the development of an FPV drone as a weapon — this is recognized in Ukraine, Russia, the West, and China. One of the developers told Oborontsi that he releases new software updates every few weeks to make his product show better results.
The only question is whether machine vision and “drone swarm” technologies will have time to mature by the end of this war or will they be able to fully manifest themselves during the next one.
And while the developers are only working on creating really high-quality machine vision, they have to fight mostly with ordinary FPV drones on radio communication with additional equipment. The anti-tank company of the 13th brigade of the NGU "Charter" mentioned in the text is collecting equipment for its FPVs so that they can reach the Russians in the Kharkiv region at an even longer distance and even prevent them from approaching Ukrainian positions. You can help the military at this link.
See also: Dragon drone, drone swarm, machine vision and fiber optic FPV