I'll throw in my two cents on this somewhat old thread.

It all comes down to harmony, and not the "chord-scale" harmony that so many people are taught. The problem with scale theory as it is normally taught is that it fails to account for the fact that not every tone in a scale is appropriate for a particular harmony (chord.) You are told "use this scale on this chord)" without any discussion of which tones need to be excluded, nor which tones are important pivots.

You can certainly learn by teaching yourself to listen and play by ear, but this is the hard and slow route. A better route is the more structured one that involves:

(a) learning how to harmonize melodies (ie create chord progressions to fit the melody)

(b) learning which tones are appropriate for each chord and which tones are not appropriate.

Although those two steps are not difficult, good luck finding instructors or books that show it to you. I only know of a couple books and have never met any music instructor who will teach it.

Some people may counter that this is a non-creative approach or that it is better to disregard the structure and play what you hear instead, but that's not true at all. In fact the opposite is true. By learning the structure, you develop a stronger ear.

Some people pick this up without formal instruction by mimicking and imitating what they hear, and this is fine because what they are actually achieving the same thing, just on an intuitive, or subconscious level, without the "verbal instruction" to guide them.

Other people who are having some difficulty with that, would benefit from the approach that shows it to you, instead of you trying to figure it out on your own. Sometimes all we need is a gentle hand to nudge us in the right direction.

Most people who go on threads asking how to improvise, fall into the latter category. This doesn't mean you have less musical ability. It just means your learning process is a little different, but you can get to the same place with the right instruction.

I started to figure out music at an early age. My godmother bought me a xylophone with a battery operated beater when I was five or so and I started picking out melodies from the TV.

When I was 12 or 13 I started playing organ and like so many back then played the blues scale for starters. I started giving that some thought and realized I could add more notes but the notes not only had to sound good but make some sense as well.

Fortunately I went to a high school where for four years I had the luxury of being a music major. Having a thorough grounding in theory makes everything so much easier.

I was a church organist when I was 15 and was forced to kill time during the service ... if you didn't improvise you'd soon run out of music. Improvising wasn't anything special in my mind, it really was just part of the job. About the same time I joined a combo and playing with others forces you to listen and copy ideas.

I met a student of Oscar Peterson when I was 17 and he impressed upon me how he worked through everything in every key. I didn't follow through on his advice until about seven years later. I then systematically worked through motifs\ideas\scales\patterns in every key and have been doing it ever since.

I incorporate improvising into my daily warm up.

When I improvise, I usually think harmonic.

I had harmonic training (classical) as a teenager; and this helps a lot.
As a student, I also learned a little bit jazz harmonics, which broadened my horizon a bit.

Today, when I improvise, I often use a set of harmonic formulas, which I vary. And sometimes, when improvising, I come up with a theme that I later turn into a composition.

I fell into the trial-and-error category, which, if you're okay with make a LOT of errors, doesn't take as long as you think.

(There is a famous business saying that goes, "If you want to be good at sales, then learn to fail faster." I think it applies.)

The pattern play and Chord play books, really gave me some places to start, as well as starting points for styles. Which was handy.

I just mimic what I hear and am able to decide what is and what is not appropriate, I also incorporate improvisation into my warm up. A good story I have is when I was on a music course recently I was playing a Mozart Sonata, and mid way through it I forgot it. I then proceeded to improvise the rest of the recapitulation.

Improvisation (or "real-time composition" ..urgh) is easy; playing exactly what's written is the hard part. I feel the majority of true improvisation is innate; when I hear or read of people pre-planning notes or harmonies...well, um, it strikes me as contradictory. Some *can't* "just play" but *can* compose quickly and complexly enough without the need for writing it down and, as such, can bear all tangible semblance of improvisation, but, um, in truth have it not be so and, perhaps as a result, insist nobody can simply "play" without planning or, at least, theoretical fundamentals; improvisation is free of this, in my mind. Um...I mean, one need not be ignorant or incapable of such processes but, at the time of improvisation, be utterly liberated of it; to move in the swells of momentism and be so utterly in the moment that most temporal notions lose their relevance and all that matters is in the now. Neither is better than the other; both have incomparable merits...but, um, please; can we try and note the difference? Both are born of the spirit of music and their paths are altogether parallel but different; they end up at pretty much the same place and it matters not which one you're on; but one you tread looking down and the other looking ahead. Practically, um, if you're really struggling to improvise (after, say, years of attempting), learn theory and listen to loads of music; then play standard repertoire and try throwing a load of ornaments and extra harmonies on it...and arpeggiate some chords...repeat sections but miss out the occasional passage, or have a voice come in a few measures late...just jump in!
Xxx

I've always been able to make up lines, improvise. I played by ear before having lessons as a youth. I have just always been able to do it. I couldn't tell you how I do it. Dave Horne's advice about working through all the keys is worth trying. Do that enough and you'll be able to play anything, I bet. Just listen a lot and cop the licks you like - then you're on your way.