It's an odd thing, but in many languages, the ordinal associated with the number "one" is not created along regular lines by just adding whatever the ordinal suffix is (in English, -th) to the number. Thus we have the following patterns (or lack thereof)
Latin: unum primus
French: un premier
German: eins erst
Likewise, Old English didn't have an ordinal corresponding to "one", so other various superlatives meaning "foremost" or "earliest" were drafted for the job: fyrst, forma, fyrmest (also formest, Northumbrian forðmest). These words were all related to the adverb "fore" meaning "beforehand" or "in front of", "fyrst" being the "fore-est", so to speak. Over the centuries, it was the word that won out as the ordinal of "one".
The Scandinavian languages also went this way: Danish and Norwegian første, Swedish först, Icelandic fyrst.
But German and Dutch did something different. Another Old English possibility for the "one-most" was "erst", the superlative of "ere" (early, soon, before). This word now survives in English only in "erstwhile" (former). But in Dutch eerste and in German erste won out as the words for "first". (Fürst, just to be confusing, is the German word for "prince", the foremost ruler.)
By the way, "1th" seems to be quite popular in Japan; the picture at the top of this post was just one of quite a few "Japlish" examples I found on Google images.
For the story on "second", see this post:
For the story on "third", see this post: