It’s not double blurring your game. Programmers can decide which part should be blurred in what strength selectively. Exactly because your eyes and brain blur in real world, the game tries to simulate the effects of blur too. In example in a racing game to get sense of speed, it can blur parts of the ground in example. If you play a racing game, your eyes do not blur the image. So its not blurred twice.

Overall its not the exact same, but it tries to simulate it like we try to simulate light in games as well.