Perhaps related to this topic, I did some tests on how the output impedance of the amplifier affects the distortion of dynamic headphones. The headphone tested was a DT880 Pro (250 Ω), using sine sweeps at 0.68 and 0.22 Vrms voltage on the drivers at 1 kHz, and with output impedances of 10.3 Ω (Xonar Essence STX headphone output) and ~110 Ω (using 100 Ω serial resistors).
Unfortunately, the measurements at low levels contain too much microphone and ambient noise. At the higher level, I repeated the tests twice, to be able to verify the repeatability of the results. The sweeps played were equalized so that the SPL and frequency response are matched at the low and high output impedance. The graphs below assume that 1 Vrms voltage on the driver at 1 kHz produces 100 dB SPL. This is probably not accurate, but the levels on each graph should be consistent nevertheless. Only the right channel was tested.
These are the results I got (click to zoom):
Frequency response: from left to right: microphone, headphone driver, amp output
THD vs. frequency: from left to right: microphone, headphone driver, amp output
THD vs. frequency: from left to right: microphone, headphone driver, amp output
Driver impedance vs. frequency:
Increasing the output impedance of the amplifier does indeed seem to increase the bass distortion of the driver, especially around its resonance frequency. This cannot be explained with the amp "working harder" (see graphs), or the headphone having to produce a higher SPL in the bass range (because it is equalized). The effect is not major, however, it is a difference of about 2 dB, but it is there. With a low impedance source that is equalized to match the frequency response of a high impedance one - which is what I did - the distortion is lower than it is with actual high output impedance.
Although the effect probably varies depending on the headphone model, and some are more affected than others. It would be interesting to try the same with orthodynamic headphones, to confirm if they are not only a more or less purely resistive load, but also fairly linear as well.