Ask a coach to evaluate a player and they will watch them. They will form opinions about touch, work rate, positioning, attitude, and potential. They will compare what they see against a mental template built over years of experience.
Then they will give you their verdict. And they will be confident about it.
The problem is that this process — the eye test — is systematically unreliable. Not occasionally wrong. Systematically, predictably wrong in ways that compound over time and embed themselves in selection decisions at every level.
The Numbers Behind the Bias
Research on observer reliability in sport — the degree to which two experienced coaches watching the same player agree with each other — consistently produces uncomfortable results. Studies across football, basketball, and talent identification contexts find inter-rater agreement rates of 50–65% for subjective assessments. Flip a coin twice and you can get close to that.
More concerning is intra-rater reliability — how consistently the same coach evaluates the same player over time. When coaches assess the same footage twice, separated by weeks, their scores shift significantly. Their baseline has drifted. Their standards have moved in response to the last player they watched.
The eye test doesn’t produce stable measurements. It produces impressions — shaped by mood, fatigue, prior exposure, and dozens of cognitive biases that operate below conscious awareness.
What Confirmation Bias Looks Like on a Pitch
Confirmation bias is the tendency to seek and interpret information in ways that confirm what you already believe. In a scouting context, it is extraordinarily powerful.
Once a coach has decided a player has potential, they notice the moments that confirm it. The crisp pass. The intelligent run. The recovery tackle. The moments that don’t fit the narrative — the hesitation, the wrong decision, the cognitive mistake — are processed differently. They become anomalies, not signals.
The reverse is equally damaging. A player who doesn’t look the part — who lacks the physical profile, the stylistic preference, the club background that the coach associates with quality — will have their positive moments explained away, and their weaknesses amplified.
Research by Williams and Reilly in talent identification in football found that physical attributes — height, build, early maturation — consistently dominated expert assessments, even when evaluators were explicitly instructed to focus on technical and cognitive qualities. The eye sees what it expects to see.
What the Eye Test Systematically Misses
The eye is well-suited to evaluating some things. It can register technical quality — first touch, passing range, shot mechanics. It can detect physical attributes — pace, strength, agility. It can notice off-ball movement and positioning patterns, at least in broad terms.
What it cannot reliably detect is cognitive load and decision quality.
When a player receives the ball under pressure and plays it quickly, is that a good decision made fast, or a rushed decision made under panic? The outcome might look the same. The mental process is radically different — and the mental process is what predicts sustained performance.
Decision latency — the time between receiving information and selecting a response — is invisible to the eye. The choice architecture — how many options were considered, how quickly they were evaluated, how consistent the selection criteria are across repeated similar situations — leaves no visible trace that an observer can detect.
These dimensions are not peripheral. They are, as the research increasingly shows, among the strongest predictors of performance ceiling in developing players.
The Case for Standardised Assessment
Standardised, objective assessment doesn’t replace expert observation. It adds a dimension that observation cannot provide.
When every player goes through the same assessment environment — the same time pressures, the same information complexity, the same measurement criteria — you get something that coaches’ reports cannot give you: comparison without context dependence.
A player assessed at a small club in northern Portugal under a pragmatic coach who values physicality gets the same cognitive-technical measurement as a player assessed at a high-profile academy with elite facilities. The score is honest because the environment is controlled.
This is the principle behind the SAC framework. Speed, Accuracy, Consistency — measured under standardised conditions, producing a score that is both precise and comparable. Not a replacement for human judgement. A corrective for human bias.
Explore the EDAC programme and see how your own cognitive-technical profile measures up → Find out more at /edac