When I tried to Add-WindowsFeature Desktop-Experience Windows replied with a stern
The source files could not be downloaded.
This is also known as error 0x800F0906 to the more unfortunate of Windows users.
Trying the same with the default gateway disconnected (so the server could not connect to the MSFT servers), it informed me that
The source files could not be found.
That is also known to experienced Windows installers as error 0x800F0907.
Google luckily finds many solutions for this error. For example, you can
Start the Local Group Policy Editor or Group Policy Management Console.
Expand Computer Configuration, expand Administrative Templates, and then select System. The screen shot for this step is listed below.
Open the Specify settings for optional component installation and component repair Group Policy setting, and then select Enabled. The screen shot for this step is listed below.
If you want to specify an alternative source file, in the Alternate source file path box, specify a fully qualified path of a shared folder that contains the contents of the \sources\sxs folder from the installation media.
Example of a shared folder path: \\server_name\share\Win8sxs
Or, specify a WIM file. To specify a WIM file as an alternative source file location, add the prefix WIM: to the path, and then add the index of the image that you want to use in the WIM file as a suffix.
Example of a WIM file path: WIM:\\server_name\share\install.wim:3
If it is applicable to do this, select the Contact Windows Update directly to download repair content instead of Windows Server Update Services (WSUS) check box.
But it won't solve the problem.
After some experimenting I found a solution that actually works.
I tried running the setup.exe on the Server 2012 R2 CD to "update" my current Server 2012 R2 installation "to" Server 2012 R2. I chose to download updates during the installation/upgrade process.
This worked and the configuration survived (including domain membership). I just had to set the IP address again.
Now I could proceed to Add-WindowsFature Desktop-Experience without error.
I used Product KeyFinder to make sure I used the same CD key for the upgrade.