Biggest European Cities that start with L on a Map

Yes, it is the same with a vs. ä and o vs. ö in Finnish and other languages using them (also u vs. ü in German and Estonian). Two dots above the letter mark that the vowel is front, not back. They are different phonemes and there are minimal pairs like Finnish "pelata" 'to play' vs. "pelätä" 'to be afraid'.

Even if I know the rule, at first it was somewhat surprising to find Finnish Äänekoski among cities that start with A or Swedish Örebro with O. But I understand that you easily drop the diacritics you don't use in your own language, as I myself always write "Zelezny" and so on.