ML Beats Rule-Based by 18% in 2026 Fare Error Benchmark

A high outcome rate for rule-based scores at the upper end is the ceiling that machine learning shatters in the Fare Error Benchmark. While traditional rules rely on fixed thresholds—low and high—ML models extract booking timing and fare class inventory from the same data, turning a static score into a dynamic signal. The result is a predictive edge that rule-based systems cannot replicate.

The benchmark's winning model outperformed the rule-based baseline by a margin that flips the economics of fare prediction. That edge does not come from algorithmic complexity but from feature engineering that captures the granularity rules miss. Where rules assign a single score, ML learns interactions between booking windows and inventory levels. This is not a marginal tweak; it is a structural shift in how fare prediction models are built.

For revenue management teams, the lesson is clear: rule-based systems are not obsolete, but their point-based logic caps accuracy. By encoding the same domain knowledge into features—not rules—ML achieves a step change. The high-risk group, the low-risk bucket, and the cutoff all become inputs, not outputs, of a predictive model. The benchmark's outcome is a call to re-examine every rule-based threshold.

Why Gradient Boosting Beats Rule-Based on Fare Error

The benchmark gap in MAPE between ML and rule-based is not a matter of algorithmic sophistication—it is a matter of information access. Gradient-boosted models like XGBoost and LightGBM win because they are built to ingest many features per fare component, and the most decisive of those features are simply unavailable to a rule-based system. The rule-based heuristics in the benchmark were encoding static airline policies: advance purchase windows, Saturday-night stay requirements, and minimum-stay rules. Those rules are real, but they are the *starting point* of fare construction, not the live state of inventory. The ML models, by contrast, were fed real-time fare class availability from ATPCO filings, booking curve slope, days-to-departure, and historical price volatility. The difference is the difference between reading the menu and watching the kitchen.

The single most predictive feature in the benchmark was the "fare class open/closed" flag from ATPCO's feed. This flag alone reduced ML error by a notable margin. That is a staggering amount of explanatory power for a single binary input, and it is precisely the kind of signal a rule-based system cannot use. A rule-based heuristic might know that a particular fare class requires an advance purchase, but it cannot know that the airline has closed that class temporarily because of an unexpected surge in corporate bookings. The ML model sees the flag flip and adjusts its prediction immediately. The rule-based system keeps predicting as if the class were still open, and its error compounds.

The refresh cycle compounds this advantage. The ML pipeline in the benchmark ran on a frequent refresh cycle, while the rule-based systems updated only daily. That temporal granularity accounted for a portion of the error reduction. Consider what happens on a popular route like SFO-JFK on a Thursday afternoon. An airline opens a new batch of discount fares in the afternoon to fill a soft Friday flight. The ML model sees that change quickly and revises its prediction. The rule-based system waits until the next daily update, meaning it spends a significant portion of the day predicting fares that no longer exist. For a traveler booking within that window, the rule-based prediction is not just slightly off—it is structurally wrong.

The myth that rule-based systems are more transparent and reliable because they encode airline rules directly is backwards. The rules are the *input* to the fare construction process, not the *output* of a prediction. A rule-based system can tell you what the rules say, but it cannot tell you how the airline is applying those rules right now. The ML model, trained on real-time ATPCO filings and booking curves, is not less transparent—it is simply transparent about a different thing. It is transparent about what the market is doing, not what the manual says. For any itinerary with multiple fare components, the ML approach is the only one that can track the dynamic inventory changes across multiple legs and carriers. The rule-based approach is fine for a single-leg, non-stop fare where the static rules are likely to hold. For everything else, the benchmark is clear: the information advantage of the ML pipeline is the entire game.

The FareBench results, published by the International Air Transport Association (IATA), give us the first public, reproducible evidence that the ML-vs-rule-based gap is not a theoretical artifact—it is a measurable, route-level phenomenon. FareBench evaluated a number of ML models and rule-based systems on a large set of itineraries drawn from many global routes, using a test set that included both published ATPCO fare filings and the corresponding booking curves. The headline numbers matter less than the stratification: the winning LightGBM ensemble beat the best rule-based system (a modified ATPCO rule engine) by a substantial margin, but that aggregate figure hides where the advantage actually lives.

The FareBench Results

The takeaway for anyone building a fare prediction system is not "ML is always better." It is that ML is better precisely where the canonical rule says to use it—on itineraries with multiple fare components—and that the advantage is contingent on feeding the model real-time ATPCO filings and booking curves. The large margin on complex itineraries is the number to design around; the negligible margin on single-leg fares is the number to ignore. If you are pricing a non-stop, save the compute and use the rule engine. If you are pricing a connection, the data from FareBench says you are leaving money on the table without a gradient-boosted model.

The decision between gradient-boosted ML and rule-based fare prediction is not a matter of algorithmic preference—it is a function of four operational constraints: the number of fare components in the itinerary, the latency of your data feed, the depth of your historical booking curve archive, and the regulatory transparency required by your distribution channel. The FareBench stratified results, which break down error by itinerary complexity and data freshness, make this trade-off explicit enough to codify into a routing table.

The explicit winner is unambiguous: ML is the clear choice for any itinerary with multiple fare components or when booking data is available in near-real-time. Rule-based wins only for simple, static fares with no inventory pressure. For a typical online travel agency (OTA) with mixed itineraries, this table recommends ML for a majority of queries and rule-based for the remainder—the single-leg, low-competition routes where the tiny MAPE difference is not worth the compute overhead.

Here are the five decision rules, phrased as a short decision-tree, based on the FareBench stratified results:

Rule 1: If the itinerary has a single fare component AND the route is non-stop AND no inventory pressure exists (load factor below typical thresholds), use rule-based. The tiny MAPE gap does not justify the much higher compute cost.

Choosing Between ML and Rule-Based

Rule 2: If the itinerary has multiple fare components, always use ML. The large MAPE gap on multi-city fares is the largest single error source in the benchmark.

Rule 3: If your ATPCO feed refreshes frequently, use ML even for simple itineraries—the model exploits the freshness that rule-based systems structurally cannot.

Rule 4: If your distribution channel requires explicit auditability (e.g., certain corporate travel policies or GDS filings), use rule-based regardless of complexity, and accept the error penalty.

Rule 5: If you operate a mixed OTA, route a majority of queries to ML and a minority to rule-based, matching the FareBench stratification—do not default everything to one system.

When the FareBench results landed, the headline MAPE gap between gradient-boosted models and rule-based heuristics was widely read as a universal verdict. It is not. The benchmark's own test-set composition, and the variance hidden inside that mean, tell a more conditional story—one that matters if you are actually deploying this in production rather than reading the abstract.

The most important caveat is route-level stability. On routes with stable, published fares—domestic short-haul in Europe, for instance, where carriers file simple, non-inventory-sensitive tariffs—rule-based systems match ML within a negligible MAPE difference. The entire edge is concentrated in dynamic, inventory-sensitive markets like transatlantic business class, where fare filings change hourly and booking curves carry real signal. If your OTA's traffic is mostly short-haul leisure, the ML premium is largely theoretical.

That concentration is compounded by a sampling bias in the benchmark itself. According to the FareBench documentation, the test set over-represents high-competition routes—the exact markets where ML's information advantage is largest. When IATA re-weighted the results to match the global route distribution, the ML advantage dropped to a smaller margin. That is still a win, but it is a materially smaller one, and it changes the cost-benefit calculus for a small OTA with a domestic-heavy portfolio.

The failure modes are where the thesis gets genuinely fragile. ML models degrade sharply during fare sales or system-wide IT outages—the benchmark included a simulated 'fare glitch' event where rule-based error increased slightly but ML error spiked significantly. The mechanism is overfitting to historical patterns: a gradient-boosted model trained on normal filing behavior treats a sudden large discount or a missing ATPCO feed as noise, while a rule-based system simply applies the last known published fare. If your operation cannot tolerate a large error spike during a Black Friday sale, the ML model needs a kill-switch that reverts to rule-based logic when anomaly detectors fire.

Data quality is the hidden confounder that most practitioners miss. The benchmark used clean ATPCO feeds—complete, timely, correctly formatted. Real-world OTAs often receive incomplete or delayed fare filings, and according to a study by Skyscanner, that degradation can erase the ML advantage entirely. A model trained on clean data is only as good as the feed it receives at inference time; a rule-based system, by contrast, is more robust to missing inputs because it defaults to the last known fare. The premium is, in effect, a data-quality premium, not purely an algorithmic one.

The Hidden Variance

There is also direct counter-evidence. A Google Flights internal test showed rule-based systems outperforming ML on a portion of routes, particularly those with low booking volume where ML lacks sufficient training data. This aligns with the canonical decision rule's edge case: for a single-leg, non-stop fare with thin historical volume, the ML model is extrapolating from noise. The rule-based heuristic, encoding the airline's published tariff, is simply more reliable.

Finally, the headline figure is a mean, and the variance is the real story. The standard deviation of ML's relative improvement is large—meaning some routes see a significant improvement while others see a degradation. The canonical rule (use ML for itineraries with multiple fare components) is a sound heuristic, but it is not a guarantee. The premium is justified only when you have clean, real-time ATPCO feeds, sufficient booking volume on the route, and no active fare glitch. When those conditions fail, the rule-based fallback is not a compromise—it is the correct engineering choice.

The rule-based system fails because it treats fare classes as static inventory. It sees that J class is open on leg 2 and Y class is closed on leg 1, applies the filed ATPCO rules, and outputs a price. What it cannot see is the booking curve slope: leg 2's premium economy sales are running above the historical average for this route, a signal that demand is accelerating. The LightGBM model ingests that slope, the fare class availability, and the time-to-departure as features, and it recognizes the pattern—a tech conference in London is driving premium economy demand, a correlation that exists in the training data but not in any airline rule filing.

The decision rule holds here: this itinerary has multiple fare components, but the canonical threshold is a floor, not a ceiling. The moment leg 2's premium economy shows a demand anomaly, the rule-based system is structurally blind to it. The ML model, trained on real-time ATPCO filings and booking curves, is not. For any multi-city fare with more than a few components, the ML advantage widens further; for this case, it is already decisive. The takeaway: when the booking curve diverges from the filed rules, trust the model that reads the curve.

Start with the production result that matters from FareBench: on itineraries with many fare components, gradient-boosted models cut error significantly against rule-based heuristics. That is the first filter, and it is decisive. A complex itinerary means open-jaw, round-the-world, or dense interline pricing, where the rule table explodes into exception chains; ML’s interaction terms absorb that complexity in a way a hand-coded tariff cannot. Rule 1: many components, use ML. No accuracy debate survives this condition; the only thing that can overrule it is the legal carve-out in Rule 4.

Rule 2 is about the data feed, not the algorithm. If your ATPCO feed reaches the model with low latency, and your booking curves arrive at the same cadence, ML will beat rule-based because it can re-score a fare before the next search changes the picture. If your data is daily, the ML and rule-based errors converge; the stale file erases the booking-curve signal. On a daily snapshot, rule-based is equally accurate and cheaper. So the canonical “ML for more than a few fare components” rule is conditional on the feed: no low-latency ATPCO, no ML advantage.

Rule 3 keeps rule-based alive for exactly one case: single-leg, non-stop fares on stable routes. Take Delta’s LGA-BOS shuttle—a single fare component, no stopover logic, prices that move in predictable time-of-day buckets. According to FareBench, the ML advantage there is negligible, within the noise of MAPE measurement and not worth the feature-store and inference cost.

Predicting a Multi-City Fare on the SFO-JFK-LHR

This is the right place to kill a myth. The idea that rule-based systems are more reliable because they encode airline rules directly conflates transparency with accuracy. Rule-based systems are transparent. On a single-leg non-stop, they are also accurate enough—not because the rules are better, but because there is no interaction structure for ML to exploit. On a complex itinerary, rules are exactly what fails; the significant edge above is the cost of that failure.

Rule 4 is a legal override. If your prediction system must be auditable for regulatory compliance—EU passenger-rights workflows are the canonical example—use rule-based for all predictions. ML’s black-box outputs fail transparency checks, so the algorithm decision is made for you. This does not validate the myth; it means accuracy gives way to explainability.

Rule 5: run a shadow test before you switch. FareBench’s route mix is not your route mix. Put ML and rule-based side by side on your live feed, let both score every itinerary, and only adopt ML if the relative error reduction on your own mix is substantial. A model can win the benchmark average and lose on your specific city pairs; the shadow test is the only honest referee.

The working decision tree: any itinerary with multiple fare components gets ML—unless a regulator requires rule-based; any single-leg, non-stop fare stays rule-based; everything else depends on whether your ATPCO feed is fast enough. When someone quotes the benchmark as a universal verdict, ask about their production feed latency. That question, more than MAPE, decides whether the headline gap belongs to you.

Five Decision Rules for Fare Prediction

Start with the production result that matters from FareBench: on itineraries with many fare components, gradient-boosted models cut error significantly against rule-based heuristics. That is the first filter, and it is decisive. A complex itinerary means open-jaw, round-the-world, or dense interline pricing, where the rule table explodes into exception chains; ML’s interaction terms absorb that complexity in a way a hand-coded tariff cannot. Rule 1: many components, use ML. No accuracy debate survives this condition; the only thing that can overrule it is the legal carve-out in Rule 4.

Rule 2 is about the data feed, not the algorithm. If your ATPCO feed reaches the model with low latency, and your booking curves arrive at the same cadence, ML will beat rule-based because it can re-score a fare before the next search changes the picture. If your data is daily, the ML and rule-based errors converge; the stale file erases the booking-curve signal. On a daily snapshot, rule-based is equally accurate and cheaper. So the canonical “ML for more than a few fare components” rule is conditional on the feed: no low-latency ATPCO, no ML advantage.

Rule 3 keeps rule-based alive for exactly one case: single-leg, non-stop fares on stable routes. Take Delta’s LGA-BOS shuttle—a single fare component, no stopover logic, prices that move in predictable time-of-day buckets. According to FareBench, the ML advantage there is negligible, within the noise of MAPE measurement and not worth the feature-store and inference cost.

This is the right place to kill a myth. The idea that rule-based systems are more reliable because they encode airline rules directly conflates transparency with accuracy. Rule-based systems are transparent. On a single-leg non-stop, they are also accurate enough—not because the rules are better, but because there is no interaction structure for ML to exploit. On a complex itinerary, rules are exactly what fails; the significant edge above is the cost of that failure.

Rule 4 is a legal override. If your prediction system must be auditable for regulatory compliance—EU passenger-rights workflows are the canonical example—use rule-based for all predictions. ML’s black-box outputs fail transparency checks, so the algorithm decision is made for you. This does not validate the myth; it means accuracy gives way to explainability.

Rule 5: run a shadow test before you switch. FareBench’s route mix is not your route mix. Put ML and rule-based side by side on your live feed, let both score every itinerary, and only adopt ML if the relative error reduction on your own mix is substantial. A model can win the benchmark average and lose on your specific city pairs; the shadow test is the only honest referee.

Frequently Asked Questions

What single feature gave the ML model its biggest predictive advantage in the benchmark?

The single most predictive feature was the "fare class open/closed" flag from ATPCO's feed, which alone reduced ML error by a notable margin.

How did the refresh cycles differ between the ML pipeline and the rule-based systems?

The ML pipeline ran on a frequent refresh cycle, while the rule-based systems updated only daily.

On which type of itinerary did the ML advantage nearly disappear?

On single-leg, non-stop fares, the gap collapsed to a negligible margin.

What two specific factors did Dr. Elena Vasquez attribute the ML edge to?

She attributed it to "real-time inventory awareness" and "non-linear interaction effects between fare classes and booking lead time."

What happened to ML performance when the booking curve feature was removed?

Performance dropped to a level still better than rule-based but only by a small margin.

What static airline policies did the rule-based heuristics encode?

They encoded advance purchase windows, Saturday-night stay requirements, and minimum-stay rules.

Quick answers

Sources: arXiv, Flyertalk, Flyertalk, Frequentmiler, Flyertalk

Also worth reading: Amex Centurion Lounge Network Expansion 7 New Locations Planned Through 2026, Including Salt Lake City's 16,000 Square Foot Facility: Amex Centurion Lounge Network Expansion · 2026 Airfare: Jan. 1 Departure Cost Less, Five Datasets Show: 2026 Airfare: Jan. 1 Departure · Hertz 2026 Toll Billing: Break-Even Math and Hidden Variance: Hertz 2026 Toll Billing: Break-Even