Formed in Rohnert Park, California, the band originally formed as Violent World until later changing the name to Ceremony.
In 2005, the band released the EP Ruined on Malfunction Records.
This release was followed by their critically acclaimed first full-length album, Violence Violence, which included a re-recorded version of the Ruined EP.
2008 saw the release of Still Nothing Moves You, the band's first release on Bridge 9 Records, described by Keith Carman in Exclaim! magazine as "hardcore's equivalent of Hiroshima", which placed on Billboard's Top Internet and Top Heatseekers charts in August 2008.
After touring all around the world with bands including Blacklisted, Converge, and AFI, they have released new music, including their third full-length album Rohnert Park in 2010.
In 2011, the band confirmed they had signed with Matador Records, and their final Bridge 9 release would be a covers-only EP, featuring the band's take on songs by Urban Waste, Pixies, Crisis, Eddie and the Subtitles, Vile and Wire.
In March 2012, the band released the album Zoo on Matador Records which was a departure from the harder style even more so than the Rohnert Park album was, with the new record focusing on a more pre-hardcore punk sound in the style of Wire's Pink Flag (1977) album.
On May 19, 2015, they released their second album on Matador, titled The L-Shaped Man.
This record saw them move even further into the post-punk realm with many comparing the album's sound to that of Joy Division.
Their sixth album, In the Spirit World Now, is set for release through Relapse Records on August 23, 2019