His first EP, Take me to Church, displays his unique style of what you may call “twenty-first century blues”, a genre that he has been adamant on finding for himself while everyone else wanted him to steer in another direction. He has worked with many different labels over the last few years, some of which he found to be more pop-orientated than he was comfortable with, before he set himself up at home and began to record his own music in his own way in order to find this musical identity.

Lyrically, "Take Me to Church" is a metaphor, with the protagonist comparing his lover to religion. In an interview with The Irish Times, Hozier stated, "I found the experience of falling in love or being in love was a death, a death of everything. You kind of watch yourself die in a wonderful way, and you experience for the briefest moment–if you see yourself for a moment through their eyes–everything you believed about yourself gone. In a death-and-rebirth sense."[4] The song pays tribute to the late New Atheist author Christopher Hitchens, with the line "I was born sick; command me to be well".