The Haarlem painter Maarten van Heemskerck (1498-1574) was one of the many Dutch and Flemish artists who travelled to Rome during the first half of the sixteenth century. Years after his return to the north, Heemskerck painted Self-portrait with the Colosseum, in which, as the title suggests, he combines a portrait of himself aged fifty-five with a view of the monumental ruins of the Colosseum. Heemskerck’s self-portrait could be seen as a celebration of a humanist artist at the peak of his career. In the year 1553, when the painting was made, Heemskerck’s artistic reputation was well established and he was appointed Dean of the Saint Lucas Guild and church-warden in the St. Bavo church in Haarlem. It is highly plausible that he painted his self-portrait at this particular time because he wished to celebrate his authority as an elite artist who had visited the Eternal City. But Heemskerck’s self-portrait tells a much more ambiguous story. This article will explore how meaning is created in the meeting between the portrayed artist and the ruins of the Colosseum. It will be argued that the key to understanding Heemskerck’s self-portrait is this peculiar juxtaposition of the artist’s own image with one of the most remarkable ruins of ancient Rome—a juxtaposition that points towards a tension between durability and decay, eternal life and death, present and past. Finally, it will be shown that the notion of personal memory becomes the central link through which this tension is negotiated.