Listening on multiple channels might help at busier airports. Ground, ramp, approach, departure, and enroute are all on different channels. Military aircraft have their own system. (That may have contributed to the DCA accident.)
Something like this was proposed in the late 1940s. The FAA was trying to figure out how to do air traffic control, and put out a request for proposals. General Railroad Signal responded.[1] They proposed to adapt railroad block signalling to the sky. Their scheme involved people listening to ATC communications and setting information about plane locations into an interlocking machine. The listeners were not the controllers; they just did data entry. The controllers then had a big board with lights showing which blocks of airspace were occupied, and could then give permission to aircraft to enter another block.
Then came radar for ATC, which was a much better idea.
Military aircraft are typically equipped with UHF radios (in addition to civilian VHF). Many of the same systems apply, just a different RF band. And we're in the process of adding UHF capabilities to our product as a lot of these military aircraft land at civilian airports for training exercises.
I can't imagine what would've happened if we adopted block signaling for ATC ...
You don't have to imagine. We already do in many places. The North Atlantic Tracks are essentially exactly that. Aircraft give position reports and estimates, those positions reports are used to decide whether an aircraft can climb though which levels etc.
It's also used extensively in an IFR non-radar environments. Exactly why aircraft have to cancel IFR at uncontrolled airfields in the US or under a procedural ATC service in the UK. You hear it a lot around the Caribbean and Bahamas too.
I'd be curious about what happens when the ASR fails. This is not the place to guess or AI-hallucinate. As a pilot, I can always ask "Say Again" over the radio if I didn't understand. ASR can't do that. Also, it would be pretty annoying if my readback was correct, but the system misunderstood either the ATC clearance or my readback and said NO.
In the very short term, we're deploying this tech more in a post-operation/training role. Imagine being a student pilot, getting in from your solo cross country, and pulling up the debrief will all your comms laid out and transcribed. In this setting, it's helpful for the student to have immediate feedback such as "your readback here missed this detail...", etc. Controllers also have phraseology and QA reviews every 30 days where this is helpful. This will make human pilots and controllers better.
Next, we'll step up to active advisory (mapping to low assurance levels in the certification requirements). There's always a human in the loop that can respond to rare errors and override the system with their own judgement. We're designing with observability as a first-class consideration.
Looking out 5-10 years, it's conceivable that the error rates on a lot of these systems will be super-human (non-zero, but better than a human). It's also conceivable that you could actually respond "Say Again" to a speech-to-speech model that can correct and repeat any mistakes as they're happening.
Of course, that's a long ways from now. And there will always be a human in the loop to make a final judgement as needed.
I imagine that aviation regulatory bodies have high standards for this - a tool being fully additive to existing tools does not necessarily mean that it's cleared for use in a cockpit or in an ATC tower, right? Do you have thoughts about how you'll approach this? Also curious from a broader perspective - how do you sell any alerting tool into a niche that's highly conscious of distractions, and of not just false positive alerts but false negatives as well?
There are lots of small steps on this ladder.
The first is post-operational. You trigger an alert async and someone reviews it after the fact. Tools like this help bring awareness to hot spots or patterns of error that can be applied later in real time by the human controller.
A step up from that is real-time alerting, but not to the main station controller. There's always a manager in the tower that's looking over everyone's shoulder and triaging anything that comes up. That person is not as focused on any single area as the main controllers. There's precedence for tools surfacing alerts to the manager, and then they decide whether it's worth stepping in. This will probably be where our product sits for a while.
The bar to get in front of an active station controller is extremely high. But it's also not necessary for a safety net product like this to be helpful in real time.
To me, speech to text and back seems like an incremental solution, but the holy grail would be the ability to symbolically encode the meaning of the words and translate to and from that meaning. People' phraseology varies wildly (even though it often shouldn't). For example, if I'm requesting VFR flight following, I can do it many different ways, and give the information ATC needs in any order. A system that can convert my words to "NorCal Approach Skyhawk one two three sierra papa is a Cessna one seventy two slant golf, ten north-east of Stockton, four thousand three hundred climbing six thousand five hundred requesting flight following to Palo Alto at six thousand five hundred," is nice, but wouldn't it be amazing if it could translate that audio into structured data:
{
atc: NORCAL,
requester: "N123SP",
request: "VFR",
type: CESSNA_172,
equipment: [G],
location: <approx. lat/lon>,
altitude: 4300,
cruise_altitude: 6500,
destination: KPAO,
}
Kidding aside, yes, you're exactly right. We're already doing this to a large degree and getting better. Lots of our own data labeling and model training to make this good.
This is effectively AGI.
And I've not seen anyone reputable suggest that our current LLM track will get us to that point. In fact there is no path to AGI. It requires another breakthrough in pure research in an environment where money is coming out of universities.
When, not if. The "artificial intelligence" as it is presently understood is statistical in nature. To rely on it for air traffic control seems quite irresponsible.
From a non-commercial viewpoint, I like to see when people get enthusiastic to make airspace and flying safer. From a commercial perspective, I agree with others writing here that going into a highly regulated market such as air traffic, is very hard, and I can tell you why I think so.
For example German air traffic control (DFS) publishes tools which are not directly ment for ATCOS https://stanlytrack3.dfs.de/st3/STANLY_Track3.html , so they are already covering part of this market. Then there are companies already specialised in tapping into the open data of the skies https://www.skysquitter.com/en/home-2/ (Or check https://droniq.de which is specialised in integrating drones into airspace). They are all either governmental or subsidiaries, or not directly involved in air traffic control itself.
I once built a 3D airspace app which I thought could become a commercial product, but I found it is too hard to compete with companies like DFS or Boeing (ForeFlight) and others. (I published the app for free to play around with: https://raphting.dev/confident/)
Saying that, I think I thought a lot about commercialisation of airspace products and my conclusion is that most countries have a good reason to leave air traffic control governmentally owned and continue gatekeeping for new entries. These gates are very well protected, and if it is only with high fees you need to pay to even gain access to data (like when I purchased airspace data from Eurocontrol for the 3D App).
Focusing on training or "post-ops", what I think you plan to do, is probably the more viable direction.
Like, check in with the controller but most messages are sent electronically and acknowledged manually.
I have your clearance, advise when ready to copy, then you write everything down on kneeboard with a pencil and then manually put it in the navigation system, is a little archaic.
certainly speech to text is a useful transition but in the long run the controller could click on an aircraft and issue the next clearance with a keyboard shortcut. then the pilot would get a visual and auditory alert in the cockpit and click to acknowledge.
I would hope someone at NASA or DARPA or somewhere is working on it. And then of course the system can detect conflicts, an aircraft not following the clearance etc.
Also, AM voice on VHF is not full duplex and the blocking problem is very real and could be addressed potentially
I feel like, with proper UX in the cockpit and on the controller console, making it easy to send/acknowledge the clearance, and intrusively demanding immediate acknowledgment for important messages, with the controller able to talk to the pilot if it isn't immediately acknowledged, structured messages would save time, be more accurate, allow automated checks, i.e. be a superior substitute.
UX needs a ton of work and human factors validation, and would take 20 years to implement. But if you were starting from a blank slate it seems like the way to go!
There already is: Controller Pilot Data Link Communications (CPDLC).
Get an instruction, press to confirm.
At the moment, this is only used for certain types of things (clearances, frequency changes, speed assignments, etc.) along with voice ATC.
https://en.wikipedia.org/wiki/Controller%E2%80%93pilot_data_...
This is how big operations handle clearances today, complete with integration into the FMS. The pilot simply reviews the clearance and accepts it.
1. Good overview of the technologies you are using, but what product are you planning on building or have built? I understand what you are doing and it's "extra safety over existing systems" but how does it work for the end user? Is the end user an ATC or a pilot?
2. You will find that introducing new systems into this very conservative field is hard. I've built avionics and ATC electronics. The problem isn't normally technology, it's the paperwork. How do you plan on handling this?
2. Agree
[edit] (oops, sorry, seeing your edit)
2. The regulation allows for escalating assurance levels. We'll start with low assurance (advisory) and climb that ladder. We're definitely not naive about it; this will be hard and annoying. But it's inconceivable that someone won't do this in the next 10 years. Too important.
Do ground vehicles also have GPS trackers with a radio transmitter, or do they just use normal ADS-B?
> This system is completely parallel to existing infrastructure, requires zero permission, and zero integration. It’s an extra safety net over existing systems (no replacement required). All the data we need is open-source and unencrypted.
The part where you explain that it is integrable in the existing chain of command at Airports is proof enough.
Wishing you all the best for your venture.
Why do we still rely on analog narrowband AM voice over VHF to do vital comms like air traffic control? Same way as we did in the 1940s!
We should be transcribing the messages and relaying them digitally with signatures and receipt acknowledgement.
As far as digital decoding delay is concerned, this is a negligible number if implemented correctly.
Bulk of the instrument flight training is "mindgames" anyways - you see nothing other than instruments, your "seat in the pants" is likely to cheat you..
Possibly going a step further, the state of teaching aids available to CFIs is pretty sad, with online quizzes and pre-recorded videos being the pinnacle of what I have experienced... this would be an awesome opportunity to try and build "automatic CFI" - not counting for AATD time under current rules but better than chair-flying (the process of imagining a flight and one's reactions).
Eh, I guess I can flex a little. Living in the Pacific North West, I do not have to play mind games. I can almost get IMC delivered on demand. :P
- there is opportunity for permissionless passive RF sensors like this startup shows. Imagine the pilots in the CRJ received an immediate notification that an intercepting aircraft was getting blocked with(transmitting over) ATC comms on their UHF frequency. I think this could be done without decoding the voices.
- Passive radar combined with direction-finding of the VHF/UHF voice transmissions could also be integrated as another source of high-resolution tracking data
Ignoring takeoff clearances for a moment, my limited understanding is that most traffic in and out of an airport follows a prescribed pattern: You take off, turn to some particular bearing, climb to some particular altitude, contact center on some particular frequency... etc. Listening to VASAviation, it seems like this accounts for > 80% of pilot-controller communication.
It's strange to me that, given the amount of automation in a modern airliner, these instructions aren't transmitted digitally directly to the autopilot. Instead of the controller verbally telling the pilot where to go, it seems feasible that the controller (or some coordinating software) could just digitally tell the plane where to go.
I feel like that's how you dramatically decrease workload on both ends, and then maybe there's more bandwidth to focus on those takeoff clearances (and eventually automate those as well?).
So many other aspects of flight safety have been handed over to the computer to solve, it's curious to me why a critical function like air traffic control still happens verbally over VHF.
As a pilot, I am surprised by how important audio communication is for retention and awareness. Given that my visual senses are (nearly) overwhelmed with information, I think there is a risk that moving ATC from audio to visual would simply saturate the "visual channel" of pilots.
In terms of automating coordination, it's obviously possible but it would take decades to prove its relative safety. (Aviation is extremely safe.) The system would be very fragile, unless you had 24/7 fully staffed backup human ATC, which rather defeats the purpose. Practically speaking too, planes take a long time to build, and the current system allows planes built 80 years ago to fly alongside brand new ones. The cost of abolishing the 'legacy fleet' (i.e. all current passenger aircraft) is pretty high!
In a critical function like control, you don't want to split a pilot's attention. You wouldn't want them to sometimes be monitoring a datalink system, but then also sometimes be listening to the radio for deviations. Even if it's less efficient 70% of the time, you reduce cognitive load by training a pilot to ALWAYS go to the radio for clearance and command.
Of course, there are edge cases these days where pilots use datalink for some clearance delivery before taxi and enroute, but you can see how these phases of flight (before you push back and after the auto pilot is on) are selected for very low competing load. In a live terminal environment, you want a pilot focused in one place for instructions.
Furthermore, you're correct that most pilot-controller communication falls largely within tight set of procedures, but deviations are common enough (weather, traffic, emergencies, hold patterns, taxi route, etc) that you find yourself on the radio regularly to sort it.
Last thing: pilots are allowed to say "unable" if they deem an instruction unsafe. I've personally had to do that many times (most common case for me is trying to comply with a vector instruction under VFR with a cloud in my way). VFR may seem like an edge case that commercial planes don't deal with, but again that's not always true in a terminal environment. Plenty of these planes fly visual approaches all the time. And if ATC is talking directly to the computer and not through the pilot, you lose the opportunity for the pilot to quickly and clearly decline an instruction.
It happens by humans over VHF because a lot of unpredictable things happen in busy airspace, and it would require a massive investment for machines to automate all of it.
I'm also not sure that people would accept the safety risk of airplanes' autopilots being given automated instructions by ATC over the air. There's a large potential vulnerability and safety risk there. I think there's some potential for automation to replace the role of ATC currently, but I suspect it would still be by transmitting instructions to human pilots, not directly to the autopilot.
Lastly, for such a system to ever be bootstrapped, it would still need to handle all of the planes that didn't have this automation yet; it would still need to support communicating with pilots verbally over VHF. An entirely AI ATC system, that autonomously listens to and responds by voice over VHF seems like a plausible first step though.
An intermediate step would at least be transmitting those instructions digitally and showing it on a map that the pilot can follow. There have been a number of incidents where pilots misunderstood where they were, and incorrectly followed instructions.
Of course, this still keeps the pilot in the loop and ideally they will notice if something seems weird.
I believe scaling laws will hold as we start to feed all of this context data into an integrated model. You could imagine a deep-q style reinforcement learning model that ingests layers of structured and visual data and outputs alerts and eventually commands. The main challenge I foresee here will be observability... it's easy enough to shove a ton of data into a black box and get a good answer 98% of the time. But regulation is likely to require such a system to be highly observable/explainable so the human can keep up with what's going on and step in as needed.
Looking further into the future, it's plausible the concrete structures of today with humans looking out windows will be replaced with sensor packages atop a long flagpole that stream high-res optical/ir camera data, surface radar, weather information, etc into a control room with VR layers that help controllers stay on top of busier and busier airspace.
The example in the original post is actually a good case study for why trajectory planning alone breaks down. By the time the aircraft are on a predictable collision course with each other, you've lost 10+ seconds of potential remediation time that you would've had if you detected the error when it was spoken. Those 10 seconds really matter in our airspace.
ADS-B helps with this as it's self-reporting. And systems like ASDE-X are useful to track objects once they hit the ground. But low-altitude deconfliction is a big problem.
(By the way, I believe EGPWS would take priority over TCAS anyway.)
One problem for sure is that when you're close to the ground you have to be careful about buildings, cell towers, etc. Terrain is one thing, but when you're a few hundred feet AGL, you could quickly be in the way of tall structures if a TCAS alert goes off.
Existing models I've tried just do a really terrible job at it.
I did collect a bunch of ATIS recordings and hand-transcribed ground-truth data for it a while ago. I can put it up if that might be handy for y'all.
I spent a lot of time out at PDK when I worked briefly in aircraft sales. Nice airport!
Let me work on this and come back! I think we can ship you an API for ATIS there...
Do you think your model's word error rate would decrease if it had access to raw feeds from towers or was trained on data from even more airports?
Do you have any plans to make the data searchable/discoverable in real time? Something like being able to search for any audio transmissions regarding a specific flight by number, etc., available on ADS-B exchange, Flightradar, FlightAware, etc., both in text and audio form.
Yes, especially for local things like navaids, cities, etc. Our expectation is that we'll need to fine-tune for different deployments.
> Do you have any plans to make the data searchable/discoverable in real time? Something like being able to search for any audio transmissions regarding a specific flight by number, etc., available on ADS-B exchange, Flightradar, FlightAware, etc., both in text and audio form.
Definitely! This actually already exists and we're selling it to airports today.
Relying solely on speech recognition seems a little tricky though. Either way you'll probably want to involve someone from the ATC side sooner rather than later.
One of the best parts of this job is how awesome the labeling community is. So many super qualified folks like yourself. Thank you!
I was trying to find your website to get a better understanding of it all, is this it? https://www.enhancedradar.com
If so, for most people it would be super helpful if you could put together something that explains this a bit better, with visuals, videos, testimonials, etc. I work on similar challenges in a completely different problem domain, and it takes quite a bit of material / explaining to really show what we do / what's possible, and make it feel less abstract.
How sensitive of a sensor array is necessary to trilaterate Remote ID signals and birds for aircraft collision avoidance?
A Multispectral sensor array (standard) would probably be most robust.
From https://news.ycombinator.com/item?id=40276191 :
> Are there autopilot systems that do any sort of drone, bird, or other aerial object avoidance?
The bird problem is a whole other issue. Mostly handled by PIREP today if birds are hanging out around an approach/departure path.
Computer vision here is definitely going to be useful long term.
Thermal: motors, chips, heatsinks, and batteries are warm but the air is colder around propellers; RF: motor RF, circuit RF, battery RF, control channel RF, video channel RF, RF from federally required Remote ID or ADS-B beacons, gravitational waves
Aircraft have less time to recover from e.g. engine and windshield failure at takeoff and landing at airports; so all drones at airports must be authorized by ATC Air Traffic Control: it is criminally illegal to fly a drone at the airport without authorization because it endangers others.
Tagging a bird on the 'dashcam'+radar+sensors feed could create a PIREP:
PIREP: Pilot's Report: https://en.wikipedia.org/wiki/Pilot_report
Looks like "birds" could be coded as /SK sky cover, /WX weather and visiblity, or /RM remarks with the existing system described on wikipedia.
Prometheus (originally developed by SoundCloud) does pull-style metrics: each monitored server hosts over HTTP(S) a document in binary prometheus format that the centralized monitoring service pulls from whenever they get around to it. This avoids swamping (or DOS'ing) the centralized monitoring service which must scale to the number of incoming reports in a push-style monitoring system.
All metrics for the service are included in the one (1) prometheus document, which prevents requests for monitoring data from exhausting the resources of the monitored server. It is up to the implementation to determine whether to fill with nulls if sensor data is unavailable, or to for example fill forward with the previous value if sensor data is unavailable for one metric.
Solutions for birds around runways and in flight paths and around wind turbines:
- Lights
- Sounds: human audible, ultrasonic
- Thuds: birds take flight when the ground shakes
- Eyes: Paint large eyes on signs by the runways
In "Glass Antenna Turns windows into 5G Base Stations" https://news.ycombinator.com/item?id=41592848 or a post linked thereunder, I mentioned ancient stone lingams on stone pedestals which apparently scare birds away from temples when they're turned.
/? praveen mohan lingam: https://www.youtube.com/results?search_query=praveen+mohan+l...
Are some ancient stone lingams also piezoelectric voice transducer transmitters, given water over copper or gold between the lingam and pedestal and given the original shape of the stones? Also, stories of crystals mounted on pyramids and towers.
Could rotating large stones against stone scare birds away from runways?
Airborne collision avoidance system: https://en.wikipedia.org/wiki/Airborne_collision_avoidance_s...
"Ask HN: Next Gen, Slow, Heavy Lift Firefighting Aircraft Specs?" https://news.ycombinator.com/item?id=42665458
Procedures for creating "bird on Multispectral plane radar and video" dataset(s):
Tag birds on the dashcam video with timecoded sensor data and a segmentation and annotation tool.
Pinch to zoom, auto-edge detect, classification probability, sensor status
voxel51/fiftyone does segmentation and annotation with video and possibly Multispectral data: https://github.com/voxel51/fiftyone
From https://news.ycombinator.com/item?id=43260690 :
> "The National Weather Service operates 160 weather radars across the U.S. and its territories. Radar detects the size and motion of particles in rain, snow, hail and dust, which helps meteorologists track where precipitation is falling. Radar can even indicate the presence of a tornado [...]"
"Integrated sensing and communication based on space-time-coding metasurfaces" (2025-03) https://news.ycombinator.com/item?id=43261825
https://aviationsystems.arc.nasa.gov/publications/2019/SciTe...
Curious if OP has seen this paper / project before?
What were your main takeaways from this paper?
I'm actually working on a similar ATC transcription project, but more along the lines of education / entertainment for non-pilots. I'm gonna be posting a YT video about it soon, but would love to chat if you guys are open to it!
Pitching air traffic management is going to be a years long process.
Getting certified for on-board avionics is similarly challenging.
In the meantime, we'll get better and better at monitoring the airspace system and deploy that technology into unregulated applications like post-operational roles.
We've accelerated past our capabilities and need to slow down. ATC has incentive to slot takeoffs and landings as close as possible, but that is in tension with the goal of safety.
> Air traffic control from first principles looks significantly more automated.
We have a system 'designed' by history, not by intention. The ATC environment is implemented in the physical world, everyone has to work around physical limitations
Automation works best in controlled environments with limited scope. The more workaround you have to add, the noisier things get, and that's why we use humans to filter the noise by picking important things to say. Humans can physically experience changes in the environment, and our filters are built from our experiences in the physical world.
Anyway, sorry that isn't a question.
> We've accelerated past our capabilities and need to slow down.
This is a super interesting meditation. As much work as there is to be done now, the demand for air traffic is growing and power laws are concentrating it into tight airspace bubbles. It would behoove us to figure out how to make airspace more dense without compromising safety. There's lots of good economic incentive for this.
My suggestion would be to make things as repeatable and consistent as possible. This would mean forcing some airports to change their practices to be consistent with everyone else, and forcing physical layout changes and construction. Unfortunately an app can't do that =( ... and the benefits are on the other side of a paradigm shift, so it's hard to make it happen naturally.
>> more dense
Large, high passenger capacity airliners have gone out of style but that would have been the best way to get passenger density up.
As a regular passenger and for someone with a little bit of drone autonomy experience, the fact that this is possible sounds insane to me. I just assumed there was something blocking the controllers from putting two flights on the same pathway (like little planes on a board?).
My main question is, how does the "noticing" work now? What did the Controllers see that alerted them to what was unfolding?
On a clear day though, it's probably just people looking out the window of the tower (maybe with binoculars) and realizing in horror that there's been a mistake. The "noticing" is just intuiting the trajectory of the two moving objects...
On top of your analysis of commands that might flag conflicting commands, shouldn't we also be analyzing the radar to make sure there aren't any obviously conflicting paths?
I saw your comment about pilot intentionality being the major time-saver but in a situation like the Potomac River crash, the intentionality and knowledge may have been a unintentional headfake for a system like yours i.e "Yes I see the CRJ".
Maybe it's 80-20 for you, with the Potomac situation being especially rare.
Your general point is 100% correct. It's not enough to only do the command parsing; you have to also compare that to radar + trajectory planning to get the full picture. That's exactly what we're working on now. Fortunately, the radar piece is much more deterministic.
Little known fact: some towers don't even have radar at all. They're just up in the cab with binoculars.