Fact or Fiction?

Almost every poem I've written is in first person. Most of the time, I am both the narrator and the author. And a lot of the time the poems stem from some real-life experience. Where they differ is often where my mother or wife question their authenticity, "I don't remember that" or "That's not how it happened." And I shrug and tell them, no, but what happened wasn't the poem I was writing. I let the words and ideas run away with me in a truth that is true to the poem, but not factually accurate. Does this make my writing creative non-fiction? Does it make it fiction-ish? Does it really matter? Whether my words are fact or fiction, they are ideas, concepts, stories for each reader to come to with their own interpretations. My truth is not yours and yours is not mine, etc., etc. But the truth that we find in each other's words are shaped from our own experiences. And in the end, does it really matter if it's fact or fiction if it moved you? entertained you? challenged you? I guess not.