The name was applied originally to the peninsula but it now also refers to the town. The settlement began during the Middle Ages as a fishing village, renowned for its herring industry. Thanks to its seascapes, fishermen and evening light, towards the end of the 19th century it became popular with a group of impressionist artists now known as the Skagen Painters. In 1879, the Skagen Fishermen's Association was established with the purpose of facilitating the local fishing industry through the Skagensbanen railway, which opened as a narrow-gauged railway in 1890. The modern port of Skagen opened on 20 November 1907, and with the railway connections to Frederikshavn and the rest of Denmark, tourism began to develop.
During the early 1910s, Christian X and Queen Alexandrine often visited Skagen and brought friends from other European monarchies. They built a summer residence, Klitgaarden, completed in 1914.
Between the 1930s and 1950s the town grew rapidly, with the population more than doubling from 4,048 in 1930 to 9,009 in 1955. Skagen reached a maximum population of 14,050 in 1980, after which it gradually declined. As of 1 January 2020, it has a population of 7,845. Thanks to the artistic community that still remains in Skagen, the local arts and crafts trade remains important to the income of the town with its numerous crafts shops and galleries. Chains such as the international jeweller Skagen Designs have branches in the town, and given the abundance of fresh fish coming in at the port of Skagen, seafood forms a staple in Skagen's restaurants.
St Lawrence's Church was built just outside the village at the end of the 14th century, but after it was buried in drifting sand it was replaced by Skagen Church in 1841 designed by Christian Frederik Hansen. It was redeveloped in 1909–10 by Ulrik Plesner who also designed a number of other buildings in Skagen, including Klitgaarden and the railway station. Several landmarks in the town are closely associated with the Skagen Painters who used to frequent them, including Brøndums Hotel, Skagens Museum, Michael and Anna Ancher's House, and Drachmann's House.
Skagen's first school was the Latinskole, a grammar school, which was in operation from 1549 until 1739. By the end of the 19th century, three schools had been established in Skagen, and in 1921, Skagen's Skipper School was opened to train navigators for both fishing boats and merchant ships, and in 1955, the public school Ankermedets skole was opened on Skagavej. The main gymnasium of the town, Skagen Kultur- og Fritidscenter, opened in 1972, and was later expanded with an aquatic centre and a number of smaller training facilities. Skagen's Sportscenter was completed in 1974, primarily to accommodate badminton and tennis. The local football club, Skagen Idræts Klub, was founded in 1946 and reached the Jyllandsserien, one of the lower divisions in Danish football, as their best position. The Hvide Klit Golf Club is 17 km (11 mi) south of the town.
Skagen station is the most northerly railway station in Denmark and is the terminus of the Skagensbanen. Nordjyske Jernbaner operates the local train service between Skagen, Frederikshavn and Aalborg with onward national connections by DSB. From Frederikshavn, there are ferries to Gothenburg, Oslo and Læsø. Aalborg Airport is 100 km (62 mi) southwest of Skagen.