I put this guide up, due to questions from people who aren't exactly experienced in hex editing.  Hopefully, this guide can clear a few things up :P  Although this is posted in a PSP section, this really applies to anything related to computers.
Warning: hex editing files can cause them to become corrupt (if you're not sure what you're doing).  It's a good idea to always keep a backup of the files before you edit (in fact, quite a few hex editors will make backups automatically for you, however, try not to rely on this)

Introduction
What is a Hex Editor for?
Okay, you've probably used Notepad before.  You open it up, type in some text, save it, then you can open it again all dandy and fine.  However, you may have also tried to open up EXEs or DLLs in Notepad?  If so, it'll probably look like a shrimped sushi, a butter-fried tempura and a temple full of sake...  Well not quite, but yeah, you get the idea.  Notepad can't really handle binary data.

What is a Hex Editor? [ Wikipedia Article ]
A hex editor is basically a file editor, like Notepad, except it's designed to edit binary data, whereas text editors (like Notepad) are designed to edit plain text.
A hex editor displays each byte of the file in hexadecimal format rather than the more familiar ASCII format.

What is Hexadecimal? [ Wikipedia Article ]
Simply, hexadecimal is a base 16 number system.  We use a base 10 number system (decimal) usually, however, in computing, base 16 is a lot more useful.  Hexadecimal numbers usually are preceded with "0x" to avoid confusion - eg 0x10 is the number 16.  There are various other ways of representing hex numbers, eg 10h or 10₁₆, but I'll use the 0x notation.
So how does a base 16 number system work?
In our base 10 number system, we have the digits 0 to 9.  Once we get past 9, we need to start using two digits to represent the number.  That is, at 10, we start using two digits to represent numbers.  For a base 16 number system like hexadecimal, we only start using two "digits" to represent the number after 15.  So how do we represent the numbers 10 to 15 as single "digits"?  We use letters.
Here's a short table, showing the conversion between hexadecimal and decimal:
Keep in mind how numbers work in a base 16 number system.
For example, for our base 10 number system, the number 258,654 really means:

        
So, the base 16 number 0x59E2DC would have a decimal value:
        
Of course, manually doing all these conversions is cumbersome.  You can use the Windows Calculator to convert between the number systems.  Just open the Calculator, go View menu -> Scientific, and you can switch between the two number systems via the option buttons (Hex and Dec).

Okay, so what's the point in hex editing?
Well it allows you to edit binary data without screwing stuff up like Notepad would, and do some basic (and even some more advanced) hacking. :P
Also helps you understand how things work.

Okay, so where can I get a hex editor?
There's a good list at Wikipedia: http://en.wikibooks.org/wiki/Reverse_Eng...ex_Editors

My personal favourite is UltraEdit32, however Hex Workshop is probably the most popular.  Note that both of these aren't free.  Some nice free hex editors are XVI32 and HxD.


Basic Hex Editing
I'll start off with some basic examples.  In these, I'll be using UltraEdit32.

Open Notepad, and copy the following text:

Save this as a TXT file somewhere.  Now open up your hex editor, and open the TXT file in it.  Note that with some hex editors, such as UltraEdit, you'll need to switch to "Hex View" - in UltraEdit, just press Ctrl+H to do this.
You should now have a screen like the following:


You'll notice the text that you entered is in the "ASCII Display region" - basically, this is the ASCII representation of data, and it should be nearly identical to what you would see in Notepad.

Note that each character in the ASCII Display region corresponds to one byte.  I will refer to them as bytes hereafter.

Now to explain the other two sections.
The Location part pretty much displays the location of the first character in each line.  Note that locations are in hexadecimal (recall that the suffix "h" is the same as the prefix "0x").  You should see that there are 16 (or 0x10) bytes to a line in UltraEdit32 (may be different for different hex editors).
Also note that locations start at 0x00, that is, the location of the first byte in any file is 0x00.  The location of the 2^nd byte in a file will be at location 0x01.  The location of byte number 52 will be at 0x33 and so on.  The location of a byte is also usually referred to as it's offset.
The Hex Dump part displays each byte (or character) of the file in hexadecimal.  As said previously, lowercase letter "A" is given the ASCII value 97 (0x61) - so if add/edit a byte to the letter "a" in the first line (try selecting it in the ASCII Display section), you'll notice that the Hex Dump value will correspond with the hex value "61".

So I still don't really get the point of a Hex Editor...
Try changing the first byte/character of that file to 0x00 (null character).  You'll need to click in the Hexadecimal display area, on the first hex digit, and then enter the numbers.  After you've done that, you should get something like this:

Save the file, then open it in Notepad - what do you see?
For your information, a lot of binary files make use of the Null (0x00) character.  If you try to edit these with a text editor like Notepad, all you'll get is a mess.

Okay, so what can I do with hex editing?
Well, quite a bit.  If you've read up to this point, then you're basically saying that you're interested :P
You've probably already seen stuff which requires you to hex edit something.
Hex editing will also allow you to perform some basic hacking.  I figured out the RCO format via hex editing.


Basic Hex Editing Info
Numbers, more specifically, integers, basically make up computers.  So it comes to no surprise that knowing how integers are stored will play a big role.
Computers store information in bytes, however, since a byte is made up of 8 bits, they can only accept 2⁸ = 256 different values.  Quite often, numbers are made up of multiple bytes, usually:

1 byte (Char or int8) [range = 2⁸ = 256 values]
2 bytes (Word or int16) [range = 2¹⁶ = 65,536 values]
4 bytes (DWord or int32 or long int) [range = 2³² = 4,294,967,296 values]
8 bytes (QWord or int64 or long int64) [range = 2⁶⁴ = 18,446,744,073,709,551,616 values]
(note the term "word" is not to be confused with CPU word sizes)
Numbers are typically stored in reverse byte order (little endian format) in files, for example, the 4 byte number 0xFE842615 would appear as:

Signed integers are those which can take on negative values.  The primary method used to represent signed integers is two's complement.
The range of an n-bit signed integer is -2^(n-1) to (2^(n-1) - 1).
Let's take an 8 bit integer for example.  If it were unsigned, it could store the values 0 to 255.  A signed 8 bit integer, however, stores the values -128 to 127.  How does it do this though?  Basically, the numbers 0 to 127 are stored "normally", so:
0x60 (96₁₀) still has the decimal value 96 regardless of being signed or unsigned.
However, the unsigned numbers 128 to 255 are different for signed numbers.  You can think of it with this formula:
Signed value = 256 - Unsigned value
Eg:
0xFF = 255₁₀ (unsigned value) = -1₁₀ (signed value)
0x80 = 128₁₀ (unsigned value) = -128₁₀ (signed value)
0xBB = 187₁₀ (unsigned value) = -69₁₀ (signed value)

So... when you're hex editing, you may notice some sequence of 0xFFFFFFFF.  It's possible that this could be the value 4294967295, however, it's much more likely to be a signed integer with the value -1 (a common value used to represent something "invalid" or "not present" in programming).

Floating point numbers may also appear at times in files.  You may have figured that integers are limited to whole numbers.  Floating point numbers allow decimal points.  How floats work is beyond the scope of this guide - if you're interested, see Wikipedia.
I don't think there's an easy way to edit floating point numbers in UltraEdit, unfortunately.  You can use HexWorkshop instead - select the byte at which the float starts, then at the bottom of the screen, you should be able to find and edit the floating point representation of the number.
There are two standards for floating point numbers, single precision and double precision.  Single precision floats take up 4 bytes, whereas double precision takes up 8 bytes.

Common Terms
Char - usually 1 byte
Word - 2 byte integer
DWord - 4 byte integer - probably most often used
QWord - 8 byte integer - not commonly used

Offset - refers to a location.  Can be absolute (usually the case) meaning the position in the file.  Sometimes it can be relative, which refers to a difference, eg an offset of 0x05 from the location 0x02 would give the location 0x07
Nibble - 4 bits - yes half a byte is a nibble, didn't you know that?  Since one hex digit represents 4 bits, one nibble = one hex digit.
Header - a common thing used in files - usually a section at the beginning of a file which describes various things about the body of the file
Pointer - another thing which is often used, usually appears in header sections.  A pointer holds an address value (points to a certain location in the file), and is almost always a DWord.
String - refers to a "string" of text, or just basically, text.  For example, "Hello" is an example of a 5 character string.
Null - usually means 0 - eg a null character is a Char (1 byte integer) whose value is 0 or 0x00
Signed/Unsigned - Unsigned integers are always positive, whereas signed integers can be negative.


Anyways, there's some basic hex editing info.  Hope this helped.  Suggestions etc welcome.

nice tut zinga thanks

gonna put this to use

im wondering how to wildcard replacement on topmen、vsh to miminize the focus/shadow effect   that bstronga did ;)

^ Kinda off-topic but you mean batch replace the glow/shadow icons?

Great tut zinga, this will help me when I make my first about screen !

:great:

Holy bajeezus ZiNgA!  I've been waiting my WHOLE LIFE for a tut like this!!!
(Ok, a little over-enthusiastic, but the tut's an awesome contribution nonetheless - thanks heaps)!

not using dummy pic for replacement, it's something like this, which bstronga did in vsh prx/topmenu rco

O_o, never seen that before.  He could've just nulled the label section I guess.

Thanks for comments guys :P

nice tutorial zinga, this is pretty cool
i just use the hex editor part of notepad++ :P