It's not very clear what you are trying to achieve, but hacking an existing embedded device is not a simple task, certainly not something for a beginner.
I imagine you have three options:
- Buy an existing headset and hack it.
This is difficult for an experienced embedded engineer, and would likely require a code decompiler and an oscilloscope at the least (along with the knowledge of how to use them).
- Build a new device using some kind of dev-kit (i.e. Arduino or similar)
Do-able. You will probably need to know how to solder as well as how to program. Also be aware that any device you build will look like a prototype (i.e. imagine a lunch-box with wires sticking out, not a polished consumer device). - You may want to look into the "Lilypad" range if you want small/wearable components.
- Buy an existing headset, write host software to customise the behaviour
Probably the easiest learning-curve of the three options (since you just need to learn how to code, which you have to do for #1 and #2 anyway).
This way you could have a PC (or smaller, raspberry-pi or similar) connect to the headset and perform custom actions when buttons are pressed.
If you want customisability or you just want to learn about electronics and embedded software development then you could go with option #2. If you want a device that you can actually use day-to-day then option #3 is probably the better choice.