I found this odd interaction where Garret's in the Manor Basement and keeps telling me he'll tell me when he's ready, and in the meantime I can go visit the Fight Club.
I went and did a few battles then talked to him again and he still said the same thing, but then he also appeared when I went inside the library, and I was able to start the archives quest.