He pretty much says it himself that a "travelling merchant sold those keys" and he "wonders who that might have been". Where they ended up is irrelevant. This is the basis for the OG Ringmaster theory, since the latter is supposed to have made a living out of mining shards and selling them to build a circus.
The first hypothesis cannot hold true simply due to the timeline. Anna and Noel are 10, which means his wife's death should have happened 9-10 years ago. This means that during this time he put the children in the orphanage, bought a large castle (from whom? how much money do those shards make?) became world-renounced in chess, and a pokemon gym-leader, all while training his G.Gardevoir to be a journalist.
Then it makes little sense from a psychological standpoint. If he had enough money to buy a castle and travel worldwide to play chess, why not invest same money for his children? Rumor has it of course that his connections with the Meteors preceed the children's birth, but had he been lowly miner, what good would he be to their cause except as a grunt? Also, how could he have learned about the true value of the shards/keys had he been uneducated and/or not important enough to discuss about this with Solaris or other high ranked officers of Meteor. Therefore, how does his position allow him to judge this situation accordingly? We could assume espionage and long-term promotions on his part but both cases still wouldn't explain his motives for joining and then quitting.
All we can do is speculate at this point, but that doesn't mean that when something is a bit off, we should not point it out. Even if we have to set a certain timeframe as the basis of our story, the linear paths that explain the precedents need to be logically accurate. Otherwise we come across 'loopholes'.