AI and images

AI colourising and upscaling archive photos: restoration or invention?

A motor yacht cuts through the water with a woman seated on its forward deck.
Motor yacht under way.Photograph by Robert Harper. Robert Harper Photograph Library, Archive.London. From the archive, shown for illustration.

A colourised photograph answers a question the camera was never asked. That is fine as illustration and corrosive as history, and the difference is a label that takes ten seconds to write.

Restoration puts back what the original recorded and time removed; everything else, however handsome, is the model's opinion.

A machine that colourises a black-and-white photograph is not recovering anything. The colours were never in the negative. What comes out is a prediction of what colours would be unsurprising, painted over a real record with great confidence. Upscaling is the same operation in another dimension: detail that would be unsurprising, added where the film or the scan had none. Our angle is that the usual argument, whether such tools are good or bad, is the wrong one. They are tools for making interpretations. The harm begins only when an interpretation is filed, captioned and circulated as the record, and that is a labelling failure with a labelling fix.

How does AI colourisation work, and is it accurate?

A monochrome image stores one number per point: how bright. A colour image stores three. Turning one into the other means inventing the two that are missing, and infinitely many colours share any given grey. A red coat, a green coat and a brown coat can be the same tone on panchromatic film.

The researchers who built the methods say this plainly. In Colorful Image Colorization (2016), Richard Zhang, Phillip Isola and Alexei Efros describe their goal as "hallucinating a plausible color version" of a photograph and call the problem "clearly underconstrained". Their system learned from a large number of colour photographs which colours tend to accompany which shapes and textures. Grass is green, sky is blue, skin falls in a certain range. For everything without a strong prior, such as clothing, paintwork, shop fronts or packaging, the model picks something likely.

They also measured how convincing the results were. Shown a real colour photograph and the machine's version side by side, human judges picked the wrong one on 32% of trials. That figure is a measure of plausibility. It says nothing about accuracy, and the test was not designed to. Today's tools are far more persuasive, which makes the distinction more important, not less.

So the answer to "is it accurate?" is that accuracy is not what the tool attempts. Where the true colour of a subject is known from another source, such as a surviving garment, a paint record or a colour photograph of the same scene, a skilled colourist can use it. That is research, and the software is only the brush.

Does AI upscaling add real detail?

No. A scan at a given resolution has recorded what it has recorded. An enlargement by interpolation, which is what our own image enlarger does, makes the picture bigger and smoother without pretending to know more; the tool says so on its own page. An AI upscaler goes further and synthesises texture: pores, eyelashes, brickwork, type.

The PULSE paper of 2020 is the clearest statement of what that involves. Its authors describe a method that searches for "realistic" high-resolution images "that downscale to the original" low-resolution one. In other words, many different sharp faces reduce to the same blurred face, and the algorithm returns one of them. It is a plausible member of a large family, not the person who was there.

There is an honest alternative, and film archives are lucky to have it. If a scan lacks detail, the first question is whether the original holds more. A medium-format transparency usually does: an access scan made for a web page uses a fraction of what is on the film, and a second pass at higher resolution adds detail that was recorded on the day. Going back to the object is slower than pressing a button, and it is the only kind of enlargement that adds fact.

For landscapes and fabric the invented detail is mostly harmless. For faces, signs, documents, number plates and anything else a historian might read, it is not. An upscaled crowd contains people who never existed, assembled from statistical habit.

Institutions that depend on photographs being records have drawn the line accordingly. World Press Photo's rules for its 2027 contest name AI enlarging tools as an immediate breach, explaining that they "are based on generative AI models that introduce new information to enlarge and sharpen images". Alamy, in a statement dated 13 February 2025, distinguishes retouching-style tools such as denoise, which it accepts on original photographs, from Generative Fill or anything that substantially changes an image's reality, which may fail its checks.

What counts as restoration, and what is invention?

The test we use is simple: where did the information come from?

Operation Source of the information Category
Removing dust and scratches from a scan The surrounding image, to hide a defect that is not part of the picture Restoration
Correcting a colour cast caused by dye fading The remaining dye record, rescaled Restoration
Inverting and balancing a negative The negative Restoration
Sharpening, noise reduction The image itself Enhancement, non-generative
Enlarging by interpolation The image itself Enhancement, non-generative
AI upscaling beyond scan resolution A model's training data Invention
Colourising a black-and-white photograph A model's training data, or a colourist's research Invention
Filling in a missing or damaged area with generated content A model's training data Invention
Changing an expression, removing a person The editor's wishes Falsification

The boundary between the first group and the rest is the boundary of the original. Our guide to restoring faded slides digitally stays on the near side of it: fading is damage to a colour record that existed, and correcting it approximates what was there. Even then, the Image Permanence Institute describes the result as an approximation, and the US federal digitisation guidelines list colour restoration and digital infill among edits to keep out of master files.

A historical comparison makes the point better than a rule. Between 1905 and 1915 Sergei Prokudin-Gorskii photographed the Russian Empire on glass plates, exposing each scene three times through blue, green and red filters. The Library of Congress holds 1,902 of these triple negatives and in 2004 produced colour composites from them by software. Those colours are real in the sense that matters: they were recorded, separately, at the time. A colourised photograph of 1910 and a Prokudin-Gorskii composite of 1910 may look alike on a screen. One is a measurement and the other is a guess, and only a caption can tell the viewer which.

Why does it matter if an archive photograph is enhanced?

Three reasons, in ascending order of seriousness.

It misleads the casual viewer. Colour reads as presence. A colourised street scene feels like a window, and people take from it facts it does not contain: the colour of a uniform, a car, a banner.

It pollutes the source pool. Enhanced versions are more attractive than originals and get shared more. Search for a well-known historical photograph and the colourised or sharpened copy may now outrank the scan. Image models trained on the web then learn the enhancement as if it were the period, a feedback loop we describe in why image models get the past wrong.

It compromises the archive. An institution whose own files mix records and interpretations without distinction can no longer vouch for either. That is the argument of deepfakes and the historical record: an archive's value is being a fixed point.

What labelling rule should an archive adopt?

Ours has three tiers and one sentence.

  1. Master. The scan as made, from the original, with no generative processing. Never overwritten. Its hash is recorded.
  2. Restoration. A derivative in which damage has been reduced using information from the original. Described as restored, with the steps noted.
  3. Interpretation. Any derivative containing information from a model or from an editor's inference: colourised, AI-upscaled, infilled. Always a separate file with a separate identifier, never substituted for the master in a catalogue.

The sentence is the caption label, and it travels with the picture: Colourised by software from a black-and-white original; colours are not recorded fact. Or: Enlarged using AI; fine detail is generated.

In the file, the IPTC's Digital Source Type vocabulary already has the words. A tier one scan is negativeFilm, positiveFilm or print. Tier two processing of the kind that leaves the main content unchanged is what the vocabulary calls algorithmicallyEnhanced. Tier three is compositeWithTrainedAlgorithmicMedia, defined as augmentation, correction or enhancement using a generative AI model. We go through the values in IPTC Digital Source Type for scans, composites and synthetic images. Because platforms strip metadata, the visible caption is not optional.

A digitisation contract should say the same thing in advance: which tools may touch masters (none that generate), which may be used for derivatives, and how each derivative is named.

How the Robert Harper archive approaches it

The Robert Harper Photograph Library includes black-and-white work that was black-and-white by choice: the inventory lists, for example, a folder of 560 shots described as landscapes, rustic buildings and off-grid living in 1975, on black-and-white negatives and contact sheets, described under black-and-white landscapes, 1975. Colourising those would not restore anything. It would replace a photographer's decision with a model's.

More than half of the archive is still on film and has yet to be scanned, so the tiers above can be built into the workflow from the first frame instead of being reconstructed afterwards. The inventory's counts have not yet been reconciled frame by frame, and we make no claims about the condition of individual originals before they are examined.

Scanning and restoration specialists who work to a master-first standard, and who can say exactly which of their tools are generative, are the people we would like to hear from through digitisation partners.

Frequently asked questions

Is AI colourisation of old photos accurate?

No, not in the sense of recovering the real colours. A black-and-white photograph records brightness only, and many different colours share the same grey. Colourising models predict colours that are statistically plausible for the content. The authors of a leading 2016 method describe the task as producing a plausible version and the problem as underconstrained. Plausible is not the same as true.

Does AI upscaling add real detail to a photograph?

No. The detail it adds is generated to be consistent with the low-resolution input and with what the model learned from other pictures. It may look right and be wrong, especially in faces, lettering and fine pattern. Real additional detail can only come from the original: a higher-resolution scan of the negative, transparency or print.

What is the difference between restoration and enhancement?

Restoration works from information the original recorded, undoing damage or change: removing dust, correcting a colour cast from dye fading, rejoining a tear. Enhancement adds information the original never held, such as colour in a monochrome image or detail beyond the resolution. Restoration can be checked against the object. Enhancement can only be checked against taste.

How should AI-enhanced archive photos be labelled?

Visibly and in the file. State in the caption that the image has been colourised or enlarged by software and name the original medium. Set the IPTC Digital Source Type to the value for generative edits, use a distinct identifier and filename from the master, and keep the unaltered master available. The label should travel with every copy.

Should archives use AI restoration tools at all?

They can be useful for access copies, exhibitions and education, where a labelled interpretation helps people engage with a picture. They do not belong in master files or in anything presented as the historical record. Guidelines for cultural heritage digitisation keep colour restoration and infill out of masters for the same reason.

Sources

Published by Archive.London. Corrections are welcome under our editorial policy.

Licensing the archive for AI and data projects

95,487 photographs from one photographer and one rights holder. Read the dataset card, search the 503-folder inventory, discuss a sample set under NDA, or arrange a first look as scanning proceeds.

See AI and data licensingWrite to the archive