If you dug a few feet down into the earth, for instance, let's say in France, I'm sure we would all agree that the material forming the bottom of the hole would still be "France". How far does that extend for, though?
If we took the borders right down, slicing all the way to the core of the Earth, would all the countries meet at a single point? So Australia would touch Iceland?
Obviously going by your logic it's all of them, but I'm going by somewhere you could theoretically stand at, not that standing there would be particularly comfortable.
Others brought up whatever country is closest to the waters of the Mariana Trench, but I'm not sure if that's right, probably depends on whatever arbitrary method you go by.
My guess that the country with territory closest to the center of the Earth would be one of the countries on the Arctic Circle, on the sea floor within territorial waters.
The furthest point from the center of the Earth is the summit of Mount Chimborazo in Ecuador.