RPG improvisation guide.

Preparations

Many people who try to avoid improv do so by trying to prepare everything in advance, plan out every detail of their campaign, and try to figure out everything their players might do. What many don't see is that this preparation can be toned down, streamlined, and used to improv, or at the very least give the illusion of improv.

In most games you cannot prepare for everything, no matter how hard you try your players will usually throw you for a loop. This can be avoided by asking your players to stick to the story threads, but this doesn't usually lead to the most fun experience.
Instead it's often far better to let your players do what they want, and give them only enough information for them to have a destination, but let them choose the path to that destination.

Let's say we have a party that needs to reach a castle. The most straight forward path is to take the main road, stop at a village on the way to sleep and restock, and then travel back on the main road to reach the castle. But the party might not wish to take this path, so now what? Figure out the entire map, important non-player characters (NPCs), potential monsters, and so on? Almost, but not exactly.

It's far easier to prepare elements you can throw at your party at any time. A camp of bandits, a lost temple, a society of strange creatures, and so on. Don't prepare them in full detail, simply have different sets ready. What's the rough layout of the camp? Who's in charge? What's hidden at the end? What do the inhabitants want?
By keeping things vague enough you can use these tidbits at any point in your campaign. If the party never ventures off the path you'll be able to use these elements at a later point, and this means you don't constantly have to figure out entire maps for each zone.

Preparation also helps cover any weaknesses you may have when it comes to improv. Having troubles coming up with monsters on the fly? Prepare a list of them. Having troubles coming up with names on the spot? Well, I hear there's this nifty little website that can help with that. Wink wink, nudge nudge.

Players know nothing

Another thing to keep in mind is that your players only know as much as you tell them. If you draw and plan out an entire map they'll know where everything is, which isn't necessarily a bad thing, but it could disrupt the flow of the game when they pick the wrong locations in the wrong order. Instead it's often better to simply give them a half completed map with only the essentials, and perhaps a few added landmarks. Everything else is uncharted terrain to them, which means they'll get to explore if they so choose. This alone is something many players enjoy.
But more importantly it lets you dictate the pace of the game. If they've haven't had a combat encounter for a while, and are really looking for one, not being tied to a map offers more opportunities to give them this.

It goes a step further though. If you really want your players to go to town A for an important plot point, but they instead choose to go to town B, or no town at all, they not only miss out on plot relevant information, but you've also spent time on something that might never be used. This isn't always something that can be avoided entirely, but being a GM is time consuming enough as it is, so if time can be saved why not do so?
The solution is simple of course. When your players say they're going East, supposedly away from where you planned town A to be, let them. Simply pretend town A's there, without a map or previous information your players won't know any better anyway.

Reacting to players

Reacting to players like this is often crucial in improv, but the only difference between the previous example and true improv is that the latter isn't planned, or less so anyway.
By reacting to players you not only diminish how much input you have to put in, but it also involves your players in the overall story telling process, which helps to immerse them in this world, and it makes them feel more in control. So a big part of improv is to just say yes to player requests and questions, but not to all of course. You don't want to make it too easy, and you don't want them to break the entire campaign.

These moments are often the more memorable ones as well, as they often lead to crazily executed ideas. But it can help you with more than just diminishing how much you have to improv, it can help with world building too.
When a player suggests anything you can build upon it. "Are there any religious symbols I recognise in this temple?", "Is that mercenary going to a specific building or hangout?", "Are there any historic data files on this planet?", these questions can all lead to a small dump of information. The more surprising the question is to you the better. Anything you hadn't thought of before can now be filled in. Yes, those symbols belong to the cult the big bad guy is also part of. Yes, the mercenary is going to a pub where other mercenaries hang out. Yes, there is a very old data file on that planet containing a foreshadowing warning.

By building upon your players' input it comes across as if you've planned everything ahead, without actually having to do so. They could even lead to whole new side quests, which is where those prepared events I mentioned earlier come in handy. The party decides to explore the temple further? Great, throw that society of strange creature in the basement of the church, or perhaps a pillaged tomb with stolen artifacts, or even just an old wine cellar with expensive wines if the party's due some loot.

Reuse ideas

Another element that can help with improv is to reuse ideas from previous parts of the campaign. By this I don't mean copying the entire event, location, or other element exactly, but using tidbits of each to tie things together. Perhaps an NPC shows up again in some form, perhaps a strange sound is heard again, or maybe the guardians of the tomb have finally caught up with the party.
This not only further establishes these elements as being rooted in the world, they come across as planned, even if they were inserted into the campaign right after a question of a player.

Reusing ideas is can be part of preparation, but these bits can only be prepared in advance after the campaign has been going for a while. It works especially well with NPCs the party really enjoyed, or when they only narrowly escaped an improvised enemy for example.

NPC & world motivations

Having well rounded NPCs and worlds helps immensely with improv. Knowing what the NPCs want, how the world works, what desires are common, and so on, helps with creating characters out of the blue, or with throwing existing characters or events at the players out of nowhere.
A rival group might wish to get to the same treasure your party is after, an enemy king might send spies to keep an eye on the party, a family member of a murdered victim might send assassins after the party to make them pay, whether deserved or not.

A well rounded world, even a small one, will form a basis for other elements you might have to improvise. No, this does not mean creating the entire world, and planning everything out. This isn't needed, remember? It simply means having rules for your world and a few crucial NPCs. Your party needs something to overcome after all, whatever it is is the basis of everything else you may need to improvise.

Breaks

There are often a lot of minor breaks in game play that could allow you to quickly think of the next step in the campaign. Loot distribution, the party planning things among themselves, players looking up their stats or other elements, and so on. These small lulls in the game can seem like a great moment to do some improv preparation, but they aren't always. In some cases it pays to focus entirely on the game, like when the party's planning things among themselves. If you're focused on trying to come up with the next element during these moments you might miss out on crucial information from the players, which itself could be used for improv as illustrated earlier.

Some breaks do allow for improv planning though. Bathroom, snack, and drink breaks are the obvious ones, but any moment during which your full attention isn't required, like a moment in combat during which a player is looking up stats, you could think about the next steps in your campaign.
I do think it's better to do any improv planning during bathroom and similar breaks, and staying focused on the game during actual game play, but it's different for each group and person.

Take notes

Taking notes can be incredibly helpful, to some it might even be a must. Names you came up with on the fly, random details that might be of importance later, anything that might return at a later point, write them down if you think or know you'll forget them otherwise.
It'll help keep things going more smoothly the next time they're needed. Your players might remember some of these elements for you, but you don't want to rely on them when it comes to describing a scene, it's too disruptive.

Make sure to organise your notes after each session though. After enough notes you'll end up with a short book of papers, having to look through them to find a specific piece of information is disruptive as well. You could organise them in all sorts of ways, based on locations, names, alphabetically, whichever works best for you.

Practice

I know that despite all the preparation in the world, all the tips and tricks, and everything else it can still be very daunting to actually improvise parts of a campaign. There are plenty of ways to practice this though, both on your own and with others.

If you wish to practice alone you could simply give yourself random prompts, perhaps by writing random locations, questions, and/or encounter elements on cards, and drawing them at random. The more diverse they are the better, as drawing multiple cards, and combining them will force you to think more outside of the box.
To make things easier you could use the rules of your own campaign world, similar to how it'd act as a basis for other improv elements in a real setting.
There are other methods of course, but they all boil down to giving yourself random ideas to respond to.

When you have others to practice with a whole lot of opportunities arise. Them giving you prompts, them being as difficult as possible, or even just playing games which rely entirely on improv are just a few examples.
There are many, many improv heavy role playing games out there. Fiasco, Microscope, Campfire, Dread, The Quiet Year, these are just a few I've played and enjoyed tremendously. Games like Fiasco and The Quiet Year don't even need a GM, so the improv element falls on the entire group, which could make it a lot easier on you.