A lot of good points, Macy, A few I'd like to comment on...
Are you listening to the demo with the same speakers you will use to play the piano? Very likely you will listen to the demo over poor speakers attached to your PC. Or maybe over very good speakers in a high-end hi-fi system? Neither will sound much like the speakers typically used for playing live instruments.
It is true that the playback system for live performance will affect the sound of the piano, generally in a negative way. However, a playback system will tend to effect all sounds equally. That is, if the playback system has an unfortunate peak or dip at a certain frequency, that will manifest itself regardless of which piano source you use... and unless it happens to be at a frequency near where the source piano itself has some unnatural response, it is not likely to be significantly worse on one piano than another. Also, the playback system itself will sound different in different rooms, and in many cases, you may even be playing through different systems in different rooms (i.e house systems, or system provided by different sound companies). So I would say that you're still better off finding out what piano sound you like best with a good set of headphones, or whatever i.e. start with the best source you can, and proceed on the basis that whatever you select will degrade roughly equally through whatever you end up playing through. (Except that you do want to take care that whatever you choose will sound good in mono.)
While playing we hear the nuances in our playing as the piano responds to small changes in our touch. We receive feedback from the sound while playing and adjust our playing accordingly in real time.
...
From:
http://www.forum-pianoteq.com/viewtopic.php?id=1429My live performances with sampled libraries often had notes that "stuck out" in various unintended places. For example, if a note sounded fine between velocities 88 and 94, if I happened to play with 95 velocity, the note stuck out like a sore thumb.
...
I had to go back and edit, by hand, numerous velocities that had "tripped" the next highest velocity layer ... in a piano library that sported 18 separate velocity levels.
...
libraries often require that a live performance get note velocities edited here and there, in order to produce a polished-sounding performance. ...
I agree with your premise quoted above, but it's interesting that it, in a sense, almost contradicts that quote from your next section (which you quoted to illustrate a different point, that demos can be doctored). That is, I agree, you would tend to alter your playing based on audible feedback as you play, yet it sounds like, at least with that software in that example, the player was unable to sufficiently do so to create a natural result, that he still found himself needing to edit the results despite getting feedback as he played! This also points out an issue with velocity layers... the more there are (and especially if they are not blended in some way), the more points there are of potential abrupt transition, so it can be a trade-off.
I would say the 2nd worse way to pick a software piano is to listen to the same MIDI recording played by different software pianos...A MIDI recording that was recorded with one software piano will not be optimum for another software piano. The first problem is that velocity curves for different software pianos are not the same...Secondly, when we play we adjust our playing to the piano being played.
I agree about those two problems, and would add that, to a large extent, they may be the same problem. That said, I don't think it is useless to compare pianos based on hearing the same MIDI file played through them, either. I would agree that it is not a good way to evaluate their dynamic response, as you probably would alter your touch as you played if you were hearing the actual piano sound as you were playing. But for the overall tone of the pianos, the decay envelopes, the overall realism (including the resonance effects and such), I think you can still tell a good deal even when hearing a sample generated while the player was listening to some other piano sound. Or put differently, a real piano, played badly, still sounds like a real piano... and so I think the fact that you may be hearing a sample where the player was unable to alter his touch in response to the sound would tend to result more in something that sounds like a poor performance rather than a poor piano. That's still doesn't give you enough information to know for sure that a piano will sound and play well to you when you get your fingers on it, but I think it can help rule out ones where you can be pretty sure you
won't be happy with their sound, as long as you keep in mind that you should not give too much credence to things that sound like unnatural velocity response.
(Though as you also point out, in those cases where a piano has many possible adjustments, you might not want to unfairly rule something out, either... you may just not like the settings in use when the recording was done.)