Methodology · 8 min read · June 2026

How to Measure Trading Signal Accuracy (The Honest Way)

If you've ever paid for a trading-signal service, you've seen the marketing: "73% win rate." "Up 412% this year." "Members made $4,200 last week."

What you've almost never seen is the spreadsheet. Every signal, timestamped, with the actual outcome a few hours later. Without that, the headline number is meaningless.

This article walks through the simplest, fairest methodology for measuring trading signal accuracy — and shows what 371 real signals on the SultraxAI platform actually look like, including the losses.

The metric that matters: win rate at a fixed horizon

A "trading signal" is just a directional bet: buy this, or sell this, at this time. To check whether the bet was right, you need to define two things:

What "right" means. For a BUY signal, "right" means the price went up. For a SELL, the price went down.
The horizon. Up over what timeframe? 5 minutes? 4 hours? A week? The answer changes everything.

The cleanest way to publish accuracy is to commit to fixed horizons up front and check every signal against all of them. SultraxAI publishes three:

1 hour after the signal fires. Where did the price end up?
4 hours after.
24 hours after.

Each resolved signal gets a "+1" if the direction was right at that horizon, and a "0" if wrong. Average those across all signals, multiply by 100, and you have a win rate.

Why this beats more "sophisticated" metrics, at least to start with.

You can compute Sharpe, expectancy, profit factor, drawdown, and a dozen other things. All useful. But all also game-able. Win rate at a fixed horizon is hard to hide losses inside — a 50% win rate means roughly half the signals were wrong, and there's nowhere to put the missing ones.

Why 1h, 4h, and 24h?

Different traders care about different timeframes. A day trader sizing an entry needs to know what happens in the first hour. A swing trader cares more about the next day. By publishing all three, the same dataset is useful regardless of your style.

It also lets you see signal decay. If a service's signals show 60% accuracy at 1h but only 48% at 24h, that tells you the edge dies fast — useful information for sizing positions and setting stops.

What 371 SultraxAI signals actually look like

Here's the platform-wide data from the last 30 days, pulled live from the SultraxAI signal-history database:

Horizon	Resolved signals	Win rate	Avg return (directed)
1 hour	371	50.4%	+0.12%
4 hours	342	51.1%	+0.29%
24 hours	301	49.6%	+0.41%

That 50.4% number is barely above coin-flip. Honest take: it's not a magical edge. It's slightly better than random, and only when combined with proper sizing and risk management does it produce a positive expectancy.

But notice the breakdown by direction:

Direction	Win rate @ 1h	What it means
BUY	48.3%	Slightly worse than random
SELL	52.7%	Modest edge — short setups are slightly more reliable in this dataset

And by symbol-specific concentration:

Symbol	Signals	Win rate @ 1h
BTC-USD	672	54.8%
ETH-USD	418	52.1%
NVDA	89	51.7%

BTC's 54.8% across 672 signals is statistically distinguishable from a coin flip (z-score ≈ 2.49). That's a real, if modest, edge.

The two mistakes everyone makes when measuring signal accuracy

Mistake 1: Picking the horizon after the fact

If a service shows you their "best month" or their "winning trades," they're cherry-picking. The honest version is to commit to your measurement window before the signal fires and never look back.

For SultraxAI, the 1h / 4h / 24h horizons are coded into the system. Every signal automatically gets back-checked at those three timestamps — no human in the loop, no selective publication.

Mistake 2: Ignoring sample size

A "92% win rate" across 12 signals is meaningless. With small samples, you'd expect huge swings just from random noise. The rough rule: under 100 signals, treat any win rate above 60% with skepticism. It's probably variance, not skill.

The 50.4% number above feels boring partly because it's true. With 371 signals, the 95% confidence interval is roughly 45% to 55% — wide enough that we can't even rule out a true win rate of pure chance.

Beyond win rate: what else to track

Once you have honest win rate publishing in place, the next layer of metrics worth adding:

Average winning return / average losing return. A 50% win rate with +2% wins and -1% losses is profitable. A 70% win rate with +0.5% wins and -3% losses is not.
Standard deviation of returns. A volatile signal stream produces "noisy" P&L even if the average is good.
Win rate by signal strength. If the service classifies signals (weak / medium / strong), the strong ones should outperform. If they don't, the classifier is broken.

SultraxAI publishes all of these in real time on the platform dashboard. You don't have to take anyone's word for it.

Why most signal services won't ever publish this

Because if they did, their conversion rate would tank.

A retail trader staring at "50.4% at 1h" doesn't get the same dopamine hit as "73% win rate, see our latest case study." The truth is duller than the marketing, and marketing wins the funnel.

The bet SultraxAI makes is that there's a smaller-but-better audience of traders who specifically value the honesty. We'd rather have 100 paying users who came in eyes-open than 10,000 who'll churn the moment they realize the numbers were inflated.

Try it on your own signals

If you want to apply this methodology to a service you currently use:

Log every signal the service publishes for 30 days — timestamp, symbol, direction.
For each, write down the price at 1h, 4h, and 24h after the signal fired.
Compute win rate at each horizon. Compute it by direction (BUY vs SELL) too.
If the service refuses to give you the price-after data, or if the numbers diverge sharply from their advertised "win rate," that's your answer.

Or skip the manual work and just use a platform that does it automatically. SultraxAI is free to use — the scanner, the AI chat with live indicators, and the published signal-accuracy view are all available without a paywall.

Try the platform (free)