Speaking of pack depreciation. Buildcraft is pretty worthless. All it offers is a couple of engines to produce energy (ic2/thermal does it better), item transport (applied energistics is unbeatable), and a method to auto mine (thaumcraft's arcane bore, computercraft's turtles, minefactory's stuff, redpower's block breakers & frames, god I could go on forever). BC is almost entirely depreciated the moment you start adding more packs in, it's just their Quarries auto load chunks and are pretty simply to make with no a intelligence requirement to upkeep. IC2 at least supplies methods of power and things to expend it on (armor, resource extensions, auto-processing, etc). Thermal Expansion does this as well and there are pros and cons in each. TE might break the bank on energy loops, more complicated upgrading, and it's crate storage has minimal impact with AE involved. IC2 offers new trees, armor, tools, but for some reason people bitch about Solar Panels.
Whatever you do through, do not install Gregtech. Oh, it's sell point is to make IC2 harder to "extend the game", but dear god what a f*cking pain in the ass. It's not very well made, capable of royally fucking you over (try overclocking an industrial furnace), and has a skyrocketing learning curve that without a wiki NEI tells you it has an impossible circulatory loop (I think it's the machine blocks, they require X but NEI says X is only made from ins. furances which require 34 machine blocks to build). The only positive side to Gregtech is you can use it's Industrial Centrifuge to convert Lava to Tin, Copper, Tungsten, and Electrum. Electrum in turn can be resent through to produce Silver & Gold. Works great with TE thanks to both the infinite energy loop (fixed in 3?) and ability to manufacture lava. Free ingots for all!
Now, in the other packs. Bees suck and annoying as hell without applied energistics stacking the bees for you, it also has no real return rate for use. Forestry is pretty lack luster, certain trees are impossible to create without finding key biomes and establishing a house there (aka impossible with one-claim-only servers), and it's return rate is barely functioning colored blocks (can you use upside stairs yet?). Instead, just pop in the Twilight Forest. Now not only do you have several new wood types to play with, you have actual zones for armed players to hunt for and invade. No joke, the Ghast Towers without flight take highly equipped characters to survive. TW also adds new items, weapons, armor, and some very cool blocks to play with. Like the vanishing one makes a nifty door or even piston free pit drop, hedge blocks are like cacti but can be placed next to each other and damage you while mining them. And honestly, BC's quarry crash bug makes you fight a hill for it's resources rather than cheating up some danger-free auto-mining.
I also second Thaumcraft. Very amazing pack. Consumes quite a bit researching and I could do without Vis griefing the hell out of people, but it's very unique with some very useful items. Rest I'd look at would be support. Minmap, NEI/TMI, anti-grief (if going live), iron chests, maybe flat bedrock, and so on. Stuff people tend to take for granted but forget they are separate mods. Just don't go too overboard with modpacks, Minecraft is a very sloppy game coded by what I can only presume are Mac Users. It's pretty easy for this block game to consume more resources than it takes to run two X-Box 360 games simultaneously.
Both Buildcraft and Ic2 are being deprecated by newer mods. I concede on that. But BC is used as a chassi/support for a lot of other mods, for example:
Buildcraft adds oil world-gen. Oil/Fuel are one of the most compact ways of storing/accessing power. Use it alongside with Railcraft, and you can have Liquid Fuel Boilers cooking that up into some very valuable steam you can then provide to your TE Steam Dynamos or Railcraft's Steam Engines - Or even to the Turbine, converting that steam into some IC2 power. Now ain't that sweet? You can have much more fun, creative, interesting and cool experience with IC2 if you have other mods installed. Thermal Expansion does not add a pump, or a quarry, but Buildcraft does. Not the best pump, but a ok one, and was the only pump in various modpacks.
On Forestry, Bees are DEFINITELY a worthwhile investment, IF you can manage it, and it's a hell of a lot easier when done on single-player, rather than multiplayer, especially on open servers, like you said. But they're worth it, especially if you use it alongside some extra mods specific for bees, like Magic Bees and Extra Bees, which make breeding and storing your bees much easier and hassle-free - Extra Bees machines, especifically, make it possible to have all your bee breeding done in any biome, because of the Acclimatiser and shuffling of traits. Ain't that sweet? Also, with those mods you can get nearly any resource in the game with bees, sort of like magic crops, but much more difficult to turn up to 11. With the right rig you can automate your entire base using only bees. Check out some of GenericB's older videos for some extreme bee breeding. Forestry also adds a lot of other cool stuff. The new Multifarms are very energy-efficient and while somewhat expensive, not hard to automate and a good return rate. MFR is still more efficient (time wise), but Forestry is definitely not dead but alive and kicking.
IC2 isn't the standard-start mod anymore. There are so many addons to BC and mods that use MJ's, that IC2 is simply not attractive at the start because it requires a power grid of it's own. Also, nearly everything in IC2 is doable, and most of the time done better, by other mods like TE or RC, or Mekanism. By the way, Mekanism adds it's own jetpacks. The only real thing IC2 has going for it is the Quantum Armor, the Crops system, and maybe UU Matter. But IC2 Experimental made using UU-Matter much more difficult, convoluted and complicated than it needed to be. On Gregtech... It really pisses me off, but it makes IC2 worth using again, if only because it offers a late-game that IC2 simply does not have. But i hate it. If i had to choose between IC2 and BC, i'd keep BC. OH, and i just remember another thing that makes BC VEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEERY worthwhile. Motherf**** gates. Using gate conditionals on things make them SO much easier to automate or regulate, i'm not even kidding. It would be an autopick for me just for that reason, if that was the only thing that the mod offered.
Also another reason to have all of them, is that you can make more complex systems that use all of them in different ways, challenging yourself everytime. Just doing the same super-efficient tactic gets BORING after a while, so, yeah.
I also agree on Twilight Forest and Thaumcraft 4.1 They're both excellent mods.
Thermal Expansion is purpose-built for maximum performance. BC has been improved upon a LOT, and unless you have a shitton of resources flying through their transparent pipes (which causes framerate drops), you should be ok. Ars Magica 2 adds a couple of worldgen structures, but none of it's blocks is particularly resource hungry. Thaumcraft 4 is one of the most well-made mods ever. It's perfectly balanced and it also doesn't have much of an impact in resource consumption, except in some very specific parts which consume slightly more resources, like when you're infusion crafting.
Applied Energistics is an almost lag-free and very cool storage option. Useful alternatives include of course the Iron Chests mod, and i tend to prefer the JABBA barrels mod instead of Factorization Barrels, because i rarely use anything from Factorization apart from barrels, wrath lamps, and, sometimes in very specific builds, routers. Logistics Pipes is now also an option, and it can have nice interactions with Applied Energistics, but i'd advise against relying solely on Logistics Pipes for your sorting because it does tend to increase the amount of resource consumption.
IC2 is a nearly dead tech mod, since nearly everything you can do there, you can do via BC/TE. Quantum Armor might be missed, as well as the Jetpack, but both can be completely replaced by Modular Powersuits, which is better in nearly every respect in regards to Quantum Armor. Don't bother installing IC2 unless you also install Gregtech, because only then IC2 gains real value. A relatively new tech mod that has been released and is interesting is RotaryCraft - downside is it has it's own power system that works out of RL principles of energy provided in terms of torque and speed applied to gears or gearboxes, which makes it rather difficult to master. It's approach is much different from the other tech mods, as it strives to provide realistic energy generation scheme and transfering of power. Upside is that it offers one of the best ore-to-final product rates. Above 5x yield for some ores. But it is SLOW unless you provide the machines with a crapton of power via torque/speed increase. Mekanism is an interesting tech mod, and different enough that in general it is worth a shot - it also provides up to 5x Ore-to-Final-Product ratio, and it IS compatible with TE/BC power - But also has it's own power (Joules, different from MJ - Minecraft Joules), since it's part of the universal electricity API. But you won't even notice that since the machine will happily accept whatever power you provide it with and work just fine - be it RF or MJ energy. What it also does, is bring it's own jetpacks out into the table - one more nail in the IC2 coffin. It's quarry is also the most cool looking one, just saying. Digital Miner is a beast.
A server shouldn't need much power to handle these mods i outlined, especially if it's only for a few people - max of 10~12 simultaneously online.
In servers, what you need to look out for, is tick-rate lag (caused most often by having a crapton of tile entities updates, like machine animations, items flowing in pipes, light updates, redstone updates, often in chunloaded areas), and framerate lag, which is caused by rendering animations and graphics, and is normally clientside. As long as the server has a high uptime, server lag won't be an issue.
Another good tip if you're concerned about resource consumption: ExtraUtilities offer better solutions to pumping lava and quarrying - the Enderthermic Pump and the Ender Quarry. The first, when pumping lava source blocks out, will replace them with smoothstone therefore minimizing the amount of flowing lava blocks and the rendering created by them to induce lag. The second, replaces any mined blocks with dirt, so you don't completely destroy the terrain and preserve natural caves, etc. It's pipes and sorting system are also pretty good and there's just tons of useful blocks, like the Angel Block and the sound mufflers. If you haven't heard of it, check it out. Once you have a modlist let me know and i can maybe make a few suggestions.
Tentative List:
Thaumcraft 4
Ars Magica
Buildcraft 2
ThermalExpansion
Forestry
ExtraBees
ThaumicBees
Mekanism
NotEnoughItems
Applied Energistics
Tropicraft
...Mod that makes all the rubber/ copper/ tin/ etc interchangeable.
...Some sort of minimap?
I'd add in:
Tinkerer's Construct - Great scaling on tools, very useful tools that don't break when they're dead, a VERY efficient way of ore-doubling, and some UNIQUE blocks like the Drawbridge and it's Signal Bus/Signal Terminal wireless redstone transmitting. Very useful and cool. IMO it's a must have. I feel bad when i don't have it.
Railcraft (Very cool track things, and very cool and efficient way of generating energy).
Redstone Arsenal - Adds Flux-Weapons and tools and some other cool stuff.
Open Blocks - Just loads of useful blocks
Extra Utilties - Just too many useful things to pass by. No, seriously, this mod is awesome. It even adds it's own dimension that i'll leave to you to find out... But it's dark, and it's deep. Very dark.
Forestry - Cool farms, bees, an interesting starting engine, and BEES and TREES.
Extra Bees - Makes beekeeping much saner at the cost of vast amounts of power.
Magic Bees - Since you have magic mods, why not have magic bees? SWEET!
Biomes O' Plenty - Very interesting terrain generation, MANY blocks added, many little useful bits and pieces.
Carpenter's Blocks - It makes building in Minecraft MUCH more fun. Adds in SLOPES! STAIRS! And all kinds of nifty building gadgets.
Chisel - Of course. It multiplies by infinity the amount of options you have to play with to build your den of evil.
Project Red - It's basically Redpower 2, but all new code, and most of the functionality. Sadly, no Thermopiles, no Pump of Doom, and no frames...
Calclavia MFFS - Who cares about frames? Use forcefields to move entire rigs much easier than with frames. Also a way to protect your base against invaders, and many other things.
Steve's Carts - Seriously, it adds so much cool stuff for you to do with it. Player and Item detectors, modular minecarts... You can automine with this and it'll even lay tracks down for you to go through after it's done. Or you can make a tree farm. Or ALL kinds of crazyness. Awesome synergy with Railcraft, too!
Logistics Pipes - They're back. People who played 1.2.5 knows thi mod was the bee's knees of automation. It made a comeback, but now we have AE, and that's sad because AE's so good it does everything... Well, AE is expensive and requires power. Logistics Pipes are much less expensive (Basic ones, at least), AND the mod interacts very well with AE.
Minefactory Reloaded - SIMPLE, STRAIGHTFORWARD, IN YOUR FACE automation is all this mod is about. Ver powerful blocks, with very steep energy costs, but most of the time very worthwhile. Greatly simplifies mobfarms.
and, finally, one of the newest mods:
Factory Manager by VSWE (creator of Steve's Carts. What this mod does, you might think? Well, it has a thing called an inventory cable, and if you run it across every one of your machines, you'll be able to interact with them remotely from any inventory factory manager, creating conditionals, setting triggers, and sending things where they are needed. Check it out in spotlights. I think it's still balanced on the weak side because of how powerful it is, but, still... It's amazing, you can connect your entire base with it and not have dedicated machine runs and deal with 100 covers to hide your piping and whatnot. It's crazy good!
I'd also recommend some adventuring mods, like Roguelike Dungeons and the sort. Really makes overworld exploring worthwhile.
I'll plug in some more suggestions if i can remember them. I'm falling asleep in my chair... Goodnight y'all!
EDIT: All mods by Chicken Bones(creator of NEI) should be included. They're all awesome.