Adapting GANP KPI

The GANP provides detailed definitions for 23 KPIs in the areas of Efficiency, Capacity, Predictability and Safety.  While these 23 KPIs have specific definitions, they are not inclusive of all indicators that are used to support the Performance Based Approach.  The S-M-A-R-T framework should be applied when making adaption to the existing KPIs. Below are examples of different performance areas. 

For Operational Performance, some common questions will be:

Is there Delay in the system?

Delay is usually measured as an actual time against a target time.  This can be measured by departure/arrival on-time performance (KPI01/14) which is actual against a schedule time as published by an airline.  However, the KPI can be adapted to measure actual against a day of operation “Ready Time” as determined by a Flight Plan or in the automation system used to manage traffic.  Apart from direct measure of punctuality, delay can be a symptom of demand and capacity imbalance and it would be beneficial to also measure capacity KPIs such as KPI09/10/11 to determine the cause of delay in the system.

Are Flight Trajectories Efficient?

Flight Efficiency can also be useful in identifying shortfalls in the system.  There are many ways this can be defined and the PBA process will need to make sure stakeholders understand what the KPI identifies.  At a basic level, actual trajectory can be compared to a great circle distance and actual altitude can be compared to an “Ideal or Target Altitude”.  This difference will not account for safety considerations.  Over long distances, wind-optimal routes will differ from the Great Circle based approach.  Similarly with Altitude, changes will occur due to wind turbulence, other safety considerations or even the weight or optimal profile (Step Climbs) that will be largely unknown from the data sources.

Other trajectory efficiency measures will consider an Actual against a Best Achieved actual (KPI08) or Actual against a Flight Plan or Best Achieved Flight Plan.  In these cases, much of the required airspace structure required for safety is accounted and in the case of Flight Plan, it is assumed that the ATM system is considering the best available information for wind.

Given the complexity of the system, flight trajectory measures will not be perfect. However, with some effort spent in understanding their use and related causal factor information, they can be useful in identifying deficiencies and contributing to the PBA process.

Is there Efficient use of Capacity?

Poor delay performance is usually the result of Demand-Capacity imbalances.  Exceptions may include airline or passenger caused delays.  Capacity KPIs will address deficiencies yet also assess if ANSPs and Operators are making the best use of the capacity available.  They are useful for the “Review/Improve” phase of PBA where better Training or Technology Adaptations (Settings in the Traffic Flow System) can improve aviation performance. 

They require Supporting Metrics that calculate Capacity for a ATM system element, Demand and Actual Traffic using the ATM system element.  The time interval for the calculation is usually not greater than 15 minutes to mitigate effects of demand concentrations. If Actual Traffic is equal to Demand or Capacity, the ATM system is considered to be operating efficiently.  GANP KPIs provide an example of this for the airport level (KPI09, KPI10, KPI11).  However, this process can be generalized to any constrained resource, such as runway or airspace to support the specific Performance Objective.

Is the ATM system Predictable?

ICAO Doc 9854 defines predictability as “a measure of delay variance against a performance dependability target” and “the ability of airspace users and ATM service providers to provide consistent and dependable levels of performance.”  Reporting variance is a fundamental component of KPIs and Data Analytics.  Data reporting will characterize a KPI by 1) Central Tendency using Mean or Median, 2) Variance using Standard Deviation, or percentiles such as the inter-quartile range 3) Skewness including an assessment of outliers or how extreme events affect 1) and 2).

Variability/Predictability will also be targeted to the Time Horizon of the prediction.  Variability based on Airline Schedule have initial times published months before the event with times finalized often a week before the flight.  Final Flight Plan times will be established on day of operation. Capacity, demand, weather, ANSP actions also will become more predictable the closer the prediction is to the day of operation.  All can have meaning for PBA depending on the Performance Objective.

Variability can be highly influenced by events outside the control of the ANSP.  These events include weather and airline/airport actions.  Like many KPIs used to support PBA, an understanding of causal factors will significantly improve the utility of the data analytics and Review phase.

ICAO KPI01 and KPI14 provide examples of predictability of Airline Schedule Times.  KPI15 provides an indicator of flight time variability using percentile methods.

Is there Compliance with Traffic Management Assigned Times?

Compliance is defined as the difference between a Controlled Scheduled Time and an actual time of an event.  ICAO KPI03 (ATFM Slot Adherence) provides an example of one implementation applied for take-off times.  This process can be generalized to support technologies with smaller tolerance criteria as well as application to airspace locations such as a metering fix.

Like Capacity and Efficiency KPIs, these measures are used to assess the effectiveness of the ATM system on the day of operation.  They are used as part of the Review/Improve phase of PBA and will most likely lead to actions of improved training, procedures or adaptation settings used in the traffic management technology.  To balance Demand and Capacity at a particular location, traffic management systems will assign Control Times for Take-off that are calculated to minimize congestion at a location.  Capacity, Efficiency and Compliance monitoring complement each other. 

Compliance is often a first step in PBA review that identifies issues that can overload an airspace volume and result in impactful events such as un-planed Airborne Holding or Diversions.  If an ANSP is tracking impactful events and is using a Traffic Management System, Target Time Compliance will be a key element of PBA.

The ICAO KPIs are categorised into their respective KPAs.  They are further divided into 2 categories, basic KPIs (shown in the upper grouping) and advanced KPIs (shown in the lower grouping).  The intent of these 2 categories are to characterize the difficulty in accessing and processing the data and providing a performance measure.  In general, Basic KPIs can be calculated using the ACARS OOOI (OOOI Data refers to times of the actual aircraft movements of Gate Out, Wheels Off, Wheels On, and Gate In) times, coupled with Airline schedule times, whereas the Advanced KPIs require a more advanced processing of flight trajectory data or access to data that may be in Traffic Flow Management Systems that ANSPs use to assign delay (KPI07, KPI12) and balance demand and capacity (KPI09, KPI10, KPI11).