Raw payload works, but encoded version doesn't

[mmiller at hick.org (mmiller at hick.org)]

On Thu, May 17, 2007 at 10:26:05PM -0700, Whit wrote:
> I just started using Metasploit and its worked great so far.  I'm having trouble, though with the shellcode that is 
> being generated after it goes through the encoder.  I'm targeting a custom service on a VM.  I know it's vulnerable.  
> I've exploited it with and without Metasploit.  I can only get it to work with Metasploit when I use a raw payload, 
> though.
>
> The program segfaults when I use the encoded version of the payload.  I've debugged using GDB and am sure that 
> everything leading up to the shellcode execution is correct.  It's a simple buffer overflow that overwrites a return 
> address.  It returns to the proper address and continues just fine.  It always gets through the nop sled.  It just 
> doesn't ever actually execute the shellcode properly.  I've tried a bunch of different payloads and they all segfault 
> as well.
>
> Any thoughts?

Two main questions:

1. Which encoders have you tried?
2. What instruction does it crash on?