Do I need an annullment if I wasn't married in a Catholic church?

My soon to be ex-husband and I were not married in a Catholic church, and I was not Catholic at the time we got married. We never had our marriage blessed by a priest. I converted to Catholicism 3 yrs ago, and we also have a son. My ex keeps saying that I am committing a heinous sin by wanting a divorce, but is that true? Does the Catholic church even recognize our marriage? Thanks for any info!