Creating combat encounters

Type & challenge

The first thing you'll need to figure out is what kind of combat encounter do you wish to create? A small battle versus a group of zombies? A massive, city wide war? Perhaps a seemingly small fight with a suspiciously looking stranger, which then turns into a huge battle of arcane prowess.

Type and challenge go hand in hand, but there's a whole lot of wiggle room on both sides. You could have a very easy encounter, but one that is very time consuming and spans multiple waves of enemies. You could also have an incredibly difficult encounter, but only because the enemy has a massive amount of hitpoints or armor, and is therefore hard to kill.
But once you've figured out what kind of encounter you need at this point in the story, and what difficulty this challenge is, you can begin to pick a location and the monsters. Assuming you didn't already have a monster or location in mind of course.

Location

When picking monsters it's best to keep in mind the location of the encounter. You wouldn't find a water elemental in the heart of a volcano for example, or a polar bear in the desert. Picking monsters that fit the location helps with the immersion of the players, and with creating a more realistic world. Of course if your world is meant to break realism completely, then by all means put polar bears in the desert. Add some dire penguins, perhaps a terrestrial fish, and you're good to go.

Most game systems have books or other resources available that'll make picking monsters based on location, as well as challenge rating a lot easier. If you don't have any of these available to you you can always resort to mythological beings or aliens, depending on your story setting. Of course this can be difficult for some locations, as well as for some types of encounters and challenge ratings, but there's where customizing (also called homebrewing) comes in, I've written a separate guide on homebrewing monsters here.

Thinking about terrain goes way beyond making monsters fit though. The terrain can be used to the advantage of both the monsters and the characters, which could open up some potentially fun or character driven encounters. Terrain can also be used mechanically in many games, from cover to distance, and from stealth to traps.
Infiltrating a mansion becomes much harder when it has an upstairs balcony guards could watch from. An archer in the tree tops will have to be shot down by other ranged attackers, or perhaps the barbarian will just cut down the tree instead.

By using terrain differently you can turn a simple encounter on any terrain into a more uniquely flavored encounter, one that allows each character to shine in their own ways. At the same time it prevents encounters from becoming dull, even if you use the same group of fighters repeatedly. Assuming you don't use the same encounter as well of course.
You can use the same group of enemies over and over, change the location drastically in each fight, and none the of the fights will feel the same.

Picking monsters

So you know the location, you know the encounter's difficulty and type, and you know what monsters could fit in theory. So, which one should you pick? And how many? And should you scale them down, up, or leave them as is?
There's no clear answer, as it depends entirely on not just the characters, but on the players as well. Experienced players will know how to make the best use of a low level character for example, but in the hands of a new player this same character might not make it against a formidable opponent.

As far as the amount of monsters goes, think about how many players you have, and how often you want them to attack, as well as how often you want your monsters want to attack. The more monsters there are, the more often they get to attack characters, and thus the bigger the chance a character could die.
The same goes the other way, the more players you have the more they can gang up on single monsters. Add in some stun elements, and your monster might not be able to do anything at all.

Scaling monsters up and down isn't hugely important. Some might need some attack checking, just to make sure they can't one shot your characters (I've come very close to doing that), but most of the scaling will happen during the actual encounter. If there's too many low dice rolls, perhaps tune down the hitpoints a little so the fight isn't dragged out too long. If an attack is doing too much damage, perhaps subtract a few points of damage from each attack.
Little changes like these are easy to do on the fly, but some do require you to keep your dice rolls hidden. If they're not you could simply change the way the monster works, perhaps by changing or adding an attack for example. Perhaps they're out of energy, perhaps the amount of health they've lost has taken its toll, and caused the monster to enter a new phase similar to video game boss battles.
Because of this it also doesn't matter too much which monster you pick, so pick whichever one you find most interesting out of the potential candidates. If one of them can offer some lore, role playing opportunities, or other bonuses definitely go with that one though.

Moments to shine

Another thing to keep in mind when creating an encounter is whether or not it offers each player a chance to shine. Not every character has to shine in every encounter, they could take turns without them knowing it, but it is often far more enjoyable to overcome an enemy when you can do so using a character specific trait.
Those archers in the treetops I mentioned earlier are a great example of this. Dealing with them either requires people who can attack from a distance as well, or it requires something or somebody that can take down the entire tree. Either option can make the character dealing with these archers feel special, something that likely would've lacked had this encounter taken place on even ground.

While terrain once again plays an important role in this, it doesn't always have to. You can add specific weakness to tailor to specific characters. You could even add elements that boost characters or synergyze with their abilities. A flame to light arrows with against a creature weak to fire, reflecting surfaces to bounce lasers around corners, weak points in joints a blunt weapon could crush, and so on.

Story

The most important part of creating an encounter will either be terrain or story, depending on the group you're playing with. If story is the most important a whole lot of fun can be had with the encounters. Emotional battles in which beloved NPCs (non-player characters) succumb to the enemy, a humiliating defeat at the hands of a taunting opponent, secrets revealed in the midst of combat, and so on.
Ideally each encounter is story driven, but in some groups the encounter itself is all that matters. In those cases it makes creating an encounter a lot easier, as you're free to do a lot more. But let's focus on story.

Each encounter you create can be seen as a chapter in your story. It takes a while to go through, it has a big impact on the story as a whole, and (ideally) it gives the players new information about the world as a whole. A simple fight with bandits can open up all sorts of story threads. Instead of nameless NPCs they could be desperate wanderers looking for resources to survive, a cult of fanatics out to destroy all non-believers, or mercenaries hired by a force out to stop your party.
All you need are little hints the players can pick up on, which could be as simple as a dropped emblem, a single line of dialogue during combat ("Thorak didn't say they'd put up this much of a fight!"), or a plea for mercy when they're on the losing side.

Just as each chapter in a book is part of a bigger whole, so too could each encounter be part of a bigger struggle. Let's say the end goal is to overthrow a tyrannical ruler of a planet, the many encounters on the way could range from taking over a ship with the intend to reach the planet itself to stealing supplies from the enemy, and from battling the forces sent by the tyrant to battling the tyrant for the first time.

Realism

I often bring up the point of realism in my guides, which to some might be counter intuitive in inherently unrealistic games. In real life you don't take turns during combat, in real life you wouldn't be able to continue fighting after taking many wounds, and in real life a dragon would simply breath fire from the air on those pesky adventurers trying to kill it. But there are some parts of reality that can make combat more exciting, more diverse, and more memorable.

First and foremost is the element of fight and flight. In real life most beings under attack have the instinct to either fight or flee. Freezing is also an option to some beings, but it's less relevant here, although seeing how an attacking party would react to an absolutely terrified opponent could make for a very interesting encounter as well.
But back to my point, fight or flight isn't a constant. A being might attack at first, but flee as soon as it knows it can't win. The same goes for the opponents you throw at your party. Mercenaries paid to attack and/or kill the party generally won't continue when their lives are at stake. A pack of wolves looking to protect their territory or an easy prey will stop the attack once dominance has been established by the party.

It goes further though. Not every encounter is one of life and death. I covered this in the avoiding character deaths guide as well, but many fights are merely about trying to drive away an opposing force. To avoid making this guide too long I'll simply say the previously mentioned guide cover this and a lot more, so if you want to involve these elements in your play I definitely recommend checking it out.

Of course realism also means the enemy will act according to their morals and desires, and if their desire is to kill the party when given the opportunity you should go ahead with it, assuming death is an option in your player group of course. But it goes further, while giving the players chances to synergyze with the terrain's great, realistically the opponents would use those to their advantage as well.

Adding realism also means adding the opportunity to avoid combat. If something can be resolved through other means, like talking or bribes, those involved often prefer that method, especially when combat could lead to death. But a combat situation could also be settled with hand to hand combat between the two strongest fighters (a moment to shine for one character) for example.

Heroes don't always win

Your players don't have to always win every encounter, so keeping this in mind helps with creating a different range of encounters. Fleeing a lair of a dragon after accidentally waking it up, slowing down an onslaught on a village for just long enough to let the villagers escape, or trying to flee a collapsing planet while opposing forces are trying to keep you on it.
Some defeats have dire consequences, others merely hurt the party's pride, either way they offer a different form of encounters and story telling.

Strategy

Last but not least there's the element of strategy to keep in mind. Some of you might be eager to plan out entire battles with strategic elements available on both sides, using your own strategic thinking to try and outwit the players in a battle of wits, but this can suck all the fun out of the game. An encounter has to be a challenge, but it has to be able to be overcome without having to think very carefully as well.
Yes, I'm saying realism should go out of the window when it comes to deep strategical thinking. You have plenty of time to plan your campaign, thinking of ways to make your NPCs fight, and figure out ways of making combat challenging to the party, but your party does not. They have mere moments to react to what's thrown at them, add the pressure of trying not to take too long on your turn, and you quickly end up with players who probably won't see the ideal way of overcoming an encounter. If this means they pay for it it quickly feels like a punishment, one they can't exactly avoid without punishing others by thinking longer during their turns.