They are affiliated with Madame X! If you do the quiz mission in Kristiline and you complete it, the dude says that he will tell you Madame X's identity. Before he can tell you, he gets taken BY Cosmia. I think that while Cosmia isn't Madame X, she is vital to the story and will become a problem later.