What you need:
1. To accept the risk of CPU/Hardware damage/failure from overclocking
2. A good CPU cooler
3. Intel Extreme Tuning Utility
4. Real Temp
5. Intel Burn Test
6. To read some guides on overclocking Sandy Bridge CPUs such as this one (i5-2500K, the same for all intents and purposes) or on YouTube about the Visual BIOS Overview or wherever else you can find.
 
What you need to know:
The maximum CPU cores temperature is 98C (I think, but Real Temp will tell you the Tj Max in the Settings tab)..That is when the CPU hits PROCHOT and throttles down the cores..You want to keep the cores temps below 80C-85C under the most demanding tests.
 
Increasing the CPU frequency will at some point require more CPU Voltage (Vcore)..More Vcore = More heat..I prefer using an Offset Voltage rather than a fixed voltage..Offset Voltage will cause the Vcore to scale up with increased CPU load/frequency and back down when not needed..Change the CPU frequency by changing the Core Ratio for all cores (Turbo Boost ratio)..Leave Intel SpeedStep enabled.
 
As CPU load goes up, Vcore goes down..This is Intel Vdroop that helps prevent Vcore overshoot beyond Intel specs when the CPU goes from a loaded to an unloaded state..The Vdroop/Load Line Calibration setting controls that..I would set the Vdroop to one notch below the highest (least Vdroop)..The less Vdroop there is, the less the Vcore will droop/drop under load, enhancing stability..Vcore may actually increase on some settings, this can be desireable, to a certain extent, for stability during extreme overclocking, but can also cause CPU damage, so be careful with this setting and keep an eye on the Vcore under load..Do not allow Vcore to go above 1.5V under load.
 
What I would do:
1. Set all the Core Ratio to 42 (4.2GHz) for now
2. Set the CPU Voltage Offset to +.100 (1/10 volt)
3. Set the Vdroop/Load Line Calibration to one notch below max
4. Save settings and see if it boots into Windows
5. Run 10 passes of Intel Burn Test to check stability, cores temps and Vcore
 
You can substitute the above mentioned tools for any of your liking, but you need to closely monitor cores temps and Vcore under load somehow..If you can't boot into Windows, or your PC crashes when stress testing, the CPU likely needs a little more Vcore by increasing the Offset Voltage..The Intel Extreme Tuning Utility can allow you to make temporary BIOS adjustments from within Windows instead of going into the BIOS every time to make hard changes and can help with tuning things in before making hard changes, and you can make profiles too.
 
That's the best I can tell you right now, and without readily finding a manual for your motherboard and not be real familiar with your particular mobo..The Intel Visual BIOS guide in the YouTube video linked above should be close, if not the same..Don't mess with overclocking the RAM until your CPU overclock is stable..Manually set the RAM to the Corsair speed, timing and voltage specs in the BIOS, not XMP unless there is no other choice..The overclock won't increase your power usage a whole lot, Real Temp has an estimated CPU wattage monitor.
 
Find the lowest stable Vcore.

That would be your voltage offset. Wish I could help more, but your BIOS is highly unusual compared to the rest of the industry.

BTW, I figured out that if I uncheck Runtime Turbo Ratio, it displays the individual turbo ratios for each core.  And when I do this, moving the OC assistant slider does change the turbo core ratios properly.  So I'll be doing this. 
 
Anyay, I haven't overclocked my CPU at all yet, still waiting to install a new cooler as someone recommended and still researching/gathering info.

There is one alarming thing I discovered. As I said, I still haven't overclocked my CPU at all. Running at BIOS defaults for my CPU, CPU-Z reports my Core Voltage at 1.32 V under load (while running CPU-Z's own Stress CPU test from its Bench tab). It fluctuates between 1.312 V and 1.32 V but is at 1.32 V most of the time. Temps gradually rise to about 75 C. Like I said, this is at stock settings - 3.8 GHz Turbo @0 voltage offset. I didn't change anything in BIOS related to CPU except Vdroop. So I thought maybe my Vdroop change was to blame, so I changed Vdroop back to the default value of High. There was no difference - it was still at 1.32 V while running the CPU-Z stress test.

The reason this bothers me is because these Vcore values are basically overclock Vcore values for other people with the same processor. Also, since it's pretty close to the max safe Vcore, it doesn't leave me with much room to increase voltages for when I do overclock.

So, my questions are, is this normal? Is this good or bad? What Vcore values should I be getting at stock (non-overclocked) settings for my i5-2550k? Since Vcore is already so high, does this mean that I can try increasing my core multipliers without increasing voltage?