There's always a danger in writing that you can lost in world-building, and never get around to actually telling a story. This blog post by Jessica Abel is a must-read on the subject. https://jessicaabel.com/idea-debt/ I like her phrase "binders of lore."
I have my own idea debt, although it is not for a fantasy series. I think the reason I haven't done anything with it is it's a subject that I've felt doesn't work so well in prose.