Quite a lot of golfers have difficulty looking at the ball as they swing, even those who do not imagine they have any such difficulty. If you watch your companions on the course carefully you will see that some "come up off the ball" too soon on almost every shot.
One or two may even have developed what has been called "ball fright," in that they jerk upward slightly as they hit.
Either fault will alter the arc and the direction of the swing, making a near-perfect strike extremely unlikely.
But if a golfer keeps his or her head back during the swing and the swing-center quite still -- which is what looking at the ball will do for you automatically -- then there's no particular reason why each strike should not be a good one.
So to those who have even the slightest difficulty looking at the ball or "staying down on the shot" I offer this advice:
Look instead at a mark on the turf just behind, or even just in front of the ball.
This is what the golf great Harry Vardon did. He rarely looked at the ball itself. Most of the time, he wrote in one of his books, he looked at a spot half an inch behind the ball.
For practice, I suggest you drop a blob of white paint on a grass-type mat and watch the clubhead sweeping through it as you swing.
One golfer I know found that once he got used to this there was very little else wrong with his game. His handicap came down six shots in a month.